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Time  Is  Money 

p  309  R°y  L-  wood _ 

Program  managers  typically  focus  on  controlling  costs  and  delivering 
a  quality  product.  The  acquisition  stool’s  third  leg— program  schedule— 
appears  to  be  a  resource  that  can  be  slipped  to  accommodate  unstable 
funding  or  technical  difficulties.  Despite  studies  linking  high  program  cost 
and  long  schedules,  few  major  defense  acquisition  programs  are  completed 
in  less  than  a  decade.  Programs  with  longer  schedules  experience  further 
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and  Expenditure  rate  goals  over  two  decades  ago.  For  numerous  reasons, 
acquisition  program  managers  have  found  it  difficult  to  meet  established 
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“Plans  are  worthless,  but  planning  is  everything.”1 

—  Dwight  D.  Eisenhower,  1957 

The  ever-quotable  Dwight  D.  Eisenhower,  speaking 
to  a  group  of  industry  executives  who  could  be  mobilized 
for  war  at  a  moment’s  notice,  was  echoing  an  old  adage 
about  warfare:  “No  battle  plan  survives  first  contact  with  the  enemy.”2 
Eisenhower’s  message,  like  the  man  himself,  was  straightforward:  “The 
reason  it  is  so  important  to  plan  [is]  to  keep  yourselves  steeped  in  the 
character  of  the  problem  that  you  may  one  day  be  called  upon  to  solve— or 
to  help  to  solve.”  He  was  reminding  them  that  warfare  is  inherently  fluid, 
and  that  the  only  way  to  adjust  to  quickly  changing  circumstances  is  to 
have  planned  for  such  contingencies  in  advance. 

Like  warfare,  defense  acquisition  has  always  been  changeable,  but 
in  the  post-Cold  War  era  it  has  become  largely  unpredictable  from  one 
year  to  the  next.  The  fiscal,  political,  and  economic  climates  in  which  it 
now  operates  vary  so  widely  and  change  so  quickly,  that  clear-headed 
planning,  which  explicitly  accounts  for  such  variability,  is  needed  more 
than  ever. 

The  first  article  in  this  journal,  “Strategic  Planning  and  Management 
in  Defense  Systems  Acquisition”  by  Stanley  G.  Rosen,  surveys  the  land¬ 
scape  of  how  planning  tools  such  as  root  cause  analyses  and  mission/ 
strategy  mapping  are  viewed  and  used  by  defense  acquisition  profession¬ 
als.  Next,  “Improving  Program  Success  Through  Systems  Engineering 
Tools  in  Pre-Milestone  B  Acquisition  Phase,”  by  Daniel  Deitz  et  al., 
proposes  systems  engineering  methods  for  planning  in  the  early  devel¬ 
opment  phase  of  acquisition  that  widen  the  solution  space  and  improve 
risk  characterization.  Roy  L.  Wood,  in  “Time  Is  Money,”  argues  that 
“time  certain  development”  periods  on  the  order  of  6  years,  give  program 
managers  abetter  handle  on  cost  and  schedule  planning. 

The  next  article,  “Are  the  Performance  Based  Logistics  Prophets 
Using  Science  or  Alchemy  to  Create  Life-Cycle  Affordability?  Using 
Theory  to  Predict  the  Efficacy  of  Performance  Based  Logistics,”  by 
Wesley  S.  Randall,  uses  theory  to  predict  the  efficacy  of  performance 
based  logistics  and  suggests  several  lines  of  theoretical  work  from  the 
economic  and  business  disciplines  that  can  improve  predictive  capa¬ 
bilities  for  logistics  planning.  Meanwhile,  contract  planning  performed 


downrange  in  contingency  and  expeditionary  operations  should  be 
executed  by  high-level  personnel  in  Congressionally  mandated  field 
positions,  argue  E.  Cory  Yoder,  USN  (Ret.),  and  colleagues.  Finally, 
in  “OSD’s  Obligation  and  Expenditure  Rate  Goals:  A  Closer  Look  at 
the  Causal  Factor  Groupings  and  How  They  Modulate,”  Col  Robert  L. 
Tremaine,  USAF  (Ret.)  and  Donna  J.  Kinnear-Seligman  report  on  an 
Office  of  the  Secretary  of  Defense  (OSD) -initiated  study  to  examine  one 
of  the  benchmarks  used  to  help  weapon  systems  program  offices  main¬ 
tain  the  required  execution  pace  of  appropriated  funding. 

The  featured  book  in  this  issue’s  Defense  Acquisition  Professional 
Reading  List  is  J.  Ronald  Fox’s  Defense  Acquisition  Reform,  1960-2009: 
An  Elusive  Goal  (2011),  reviewed  by  John  Alic. 

There  have  been  several  changes  to  our  masthead.  First,  Dr. 
William  A.  LaPlante,  Principal  Deputy  Assistant  Secretary  of  the 
Air  Force  (Acquisition),  has  replaced  Mr.  David  M.  Van  Buren  on  our 
Research  Advisory  Board.  Second,  Mr.  Jerry  Vandewiele  has  replaced 
Dr.  Don  McKeon  as  the  DAU  Midwest  representative  on  our  Editorial 
Board.  Third,  Dr.  Neal  Couture  of  The  George  Washington  University 
and  Dr.  Aude-Emmanuelle  Fleurant  of  the  France’s  War  College 
Strategic  Research  Institute  (Institut  de  Recherche  Strategique  de 
l’Ecole  Militaire)  have  also  been  added  as  new  members  of  the  Editorial 
Board.  We  greatly  appreciate  the  service  and  continued  support  of  the 
outgoing  board  members,  which  is  critical  to  the  processing  and  pub¬ 
lication  of  the  Defense  Acquisition  Research  Journal,  and  look  forward 
to  the  contributions  of  the  newest  board  members. 


Dr.  Larrie  D.  Ferreiro 
Executive  Editor 
Defense  ARJ 


'Eisenhower,  D.  D.  (1957,  November  14).  Remarks  at  the  National  Defense 
Executive  Reserve  conference.  Online  by  G.  Peters  and  J.  T.  Wolley, 

The  American  Presidency  Project  at  http://www.presidency.ecsb.edu/ 
ws/?pid=10951. 
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Research  Agenda  2013 


The  Defense  Acquisition  Research  Agenda  is  intended  to  make 
researchers  aware  of  the  topics  that  are,  or  should  be,  of  partic¬ 
ular  concern  to  the  broader  defense  acquisition  community 
throughout  the  government,  academic,  and  industrial  sectors. 
The  purpose  of  conducting  research  in  these  areas  is  to  provide 
solid,  empirically  based  findings  to  create  a  broad  body  of  knowl¬ 
edge  that  can  inform  the  development  of  policies,  procedures,  and 
processes  in  defense  acquisition,  and  to  help  shape  the  thought 
leadership  for  the  acquisition  community. 

Each  issue  of  the  Defense  AR  J  will  include  a  different  selection 
of  research  topics  from  the  overall  agenda,  which  is  at:  http:// 
www.dau.mil/research/Pages/researchareas.aspx 


Affordability  and  cost  growth 


•  Define  or  bound  “affordability”  in  the  defense  portfolio.  What  is  it? 
How  will  we  know  if  something  is  affordable  or  unaffordable? 

•  What  means  are  there  (or  can  be  developed)  to  measure,  manage,  and 
control  “affordability”  at  the  program  office  level?  At  the  industry 
level?  How  do  we  determine  their  effectiveness? 

•  What  means  are  there  (or  can  be  developed)  to  measure,  manage, 
and  control  “Should  Cost”  estimates  at  the  Service,  Component, 
program  executive,  program  office,  and  industry  levels?  How  do  we 
determine  their  effectiveness? 


What  means  are  there  (or  can  be  developed)  to  evaluate  and  compare 
incentives  for  achieving  “Should  Cost”  at  the  Service,  Component, 
program  executive,  program  office,  and  industry  levels? 


October  2013 


•  Recent  acquisition  studies  have  noted  the  vast  number  of  programs 
and  projects  that  do  not  make  it  successfully  through  the  acquisition 
system  and  are  subsequently  cancelled.  What  would  systematic 
root  cause  analyses  reveal  about  the  underlying  reasons,  whether 
and  how  these  cancellations  are  detrimental,  and  what  acquisition 
leaders  might  do  to  rectify  problems? 

•  Do  Joint  programs— at  the  inter-Service  and  international  levels— 
result  in  cost  growth  or  cost  savings  compared  with  single-Service 
(or  single-nation)  acquisition?  What  are  the  specific  mechanisms 
for  cost  savings  or  growth  at  each  stage  of  acquisition?  Do  the  data 
support  “jointness”  across  the  board,  or  only  at  specific  stages  of  a 
program,  e.g.,  only  at  research  and  development  or  only  with  specific 
aspects,  e.g.,  critical  systems  or  logistics? 

•  Can  we  compare  systems  with  significantly  increased  capability 
developed  in  the  commercial  market  to  DoD-developed  systems  of 
similar  characteristics? 

•  Is  there  a  misalignment  between  industry  and  the  government 
priorities  that  causes  the  cost  of  such  systems  to  grow  significantly 
faster  than  inflation? 

•  If  so,  can  we  identify  why  this  misalignment  arises?  What  relation¬ 
ship  (if  any)  does  it  have  to  industry’s  required  focus  on  shareholder 
value  and/or  profit,  versus  the  government’s  charter  to  deliver 
specific  capabilities  for  the  least  total  ownership  costs? 
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Strategic  Planning 
and  Management  in 
Defense  Systems  Acquisition 


Stanley  G.  Rosen 


Strategic  Planning  and  Management  (SP&M)  methods  are 
widely  used  in  the  commercial  sector  and  are  a  required 
organizational  activity  within  the  U.S.  Government.  More 
specifically,  defense  acquisition  organizations  use  SP&M 
methods  to  strengthen  the  management  of  defense 
acquisition  organizations/programs.  This  article  reports 
results  of  a  survey  of  the  defense  acquisition  community 
that  assessed  how  SP&M  methods  and  practices  promote 
management  effectiveness.  The  results  show  that  SP&M 
is  viewed  as  valuable  to  Department  of  Defense  system 
acquisition  programs  and  organizations.  Moreover,  this 
effort  identified  high-value  activities,  tools,  processes, 
practices,  and  common  roadblocks  to  effective  SP&M. 
These  results  imply  that  training  on  processes  and  tool 
use  can  be  very  important,  especially  for  senior  leaders, 
and  implementation  assistance  can  also  be  useful. 
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For  the  purpose  of  this  effort.  Strategic  Planning  and  Management 
(SP&M)  is  a  set  of  processes  that  includes  strategic  planning,  where 
managers  jointly  formulate  their  strategy;  and  strategic  management, 
the  implementation  or  execution  of  the  strategic  plan.  These  two  pro¬ 
cesses,  formulation  and  implementation,  are  both  mutually  essential. 
Planning  without  implementation  is  useful,  but  fruitless;  implementa¬ 
tion  without  planning  is  chaotic. 

Based  on  those  definitions,  SP&M  has  the  following  key 
characteristics: 

•  Positions  the  organization  through  strategy  and  capability 
planning; 

•  Responds  to  real  time  strategic  issues;  and 

•  Tackles  systematic  management  of  resistance  during  stra¬ 
tegic  implementation. 

Strategic  planning,  according  to  Dr.  John  Bryson  (2010),  offers  many 
benefits  to  public-sector  organizations: 

•  Promotes  strategic  thinking,  acting,  and  learning; 

•  Improves  decision  making; 

•  Enhances  organizational  effectiveness,  responsiveness, 
and  resilience; 

•  Improves  organizational  legitimacy;  and 

•  Benefits  people  directly  involved. 

Bryson,  a  strategic  planning  researcher  from  the  University  of 
Minnesota,  states,  “Evidence  indicates  that  when  strategic  planning  is 
seen  as  a  practice  that  is  improved  by  reason-based  advice,  it  is  one  of  the 
very  useful  ways  in  which  imperfect  people  can  cope  pretty  well  with . . . 
‘insoluble’  problems”  (Bryson,  2010). 
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A  growing  number  of  studies  indicates  that  strategic  planning  works 
in  a  variety  of  situations,  and  that  successful  linkage  to  strategic  vision¬ 
ing,  long-range  planning,  budgeting,  and  implementation  promotes 
organizational  and  technological  innovation.  Strategic  planning  has 
become  ubiquitous  in  the  public  sector  over  the  past  25  years— with  exten¬ 
sive  practical  experience  in  managing  effective  organizational  change 
in  general,  and  with  strategic  planning  in  particular— and  has  proven 
its  value  (Barzelay  &  Campbell,  2003;  Berman  &  West,  1998;  Berry  & 
Wechsler,  1995;  Boyne  &  Gould-Williams,  2003;  Bryson,  2004;  Campbell, 
2000;  Friedman,  1987;  Mulgan,  2009;  Wechsler  &  Backoff,  1987). 

In  fact,  the  recognition  is  evolving  that  transition  is  needed  from 
strategic  planning  to  the  broader  process  of  strategic  management, 
which  focuses  the  organization  on  implementation  of  the  strategic  plan. 
According  to  Theodore  Poister  (2010),  strategic  management  promotes 
effective  strategy  implementation,  is  ongoing  rather  than  episodic, 
and  focuses  on  achieving  strategic  goals  and  objectives  rather  than  on 
measurement.  In  fact,  evidence  indicates  that  performance  monitoring 
through  measurement  informs  strategy  (Moynihan,  2008). 

The  effectiveness  of  modern  strategic  management  methodologies 
has  been  well  documented  (Eden  &  Ackerman,  1998;  Meier  &  O’Toole, 
2002;  Nutt  &  Backoff,  1992;  Poister,  Pitts,  &  Edwards,  2010).  Schmidt 
(2009)  has  written  extensively  about  the  benefits  of  applying  strategic 
management  principles  to  project  management.  These  practices  help 
address  key  issues,  including 

•  What  are  we  trying  to  accomplish  and  why? 

•  How  do  we  measure  success? 

•  What  other  conditions  must  exist? 

•  How  do  we  get  there? 
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The  work  of  Rollinson  and  Young  (2010)  identifies  key  principles  for 
successful  strategic  management  and  identifies  a  comprehensive  process 
for  the  implementation  of  these  principles.  Their  discussion  of  strategic 
management  competencies  applies  to  defense  acquisition  organizations 
and  programs: 

•  Identifying,  articulating,  and  developing  a  core  set  of  shared 
values; 

•  Visioning; 

•  Strategic  thinking; 

•  Identifying  and  developing  core  organizational  competen¬ 
cies  and  capabilities; 

•  Converting  information  into  strategic  intelligence; 

•  Identifying,  evaluating,  and  selecting  strategic  alterna¬ 
tives;  and 

•  Team  work  and  team  building. 

Situation 

The  Department  of  Defense  (DoD)  is  responsible  for  effectively  using 
taxpayer  dollars  to  field  systems  that  enhance  national  security.  And  the 
department  is  constantly  striving  to  find  ways  to  improve  performance. 
Of  course,  DoD  leadership  rightly  stresses  that  budget  reductions  are 
prompting  “doing  more  with  no  more”  (at  best).  This  is  why,  among  other 
things,  lessons  learned  from  best  practices  are  being  emphasized  (e.g., 
“Better  Buying  Power”  initiatives). 

Defense  systems  acquisition  is  inherently  a  strategic  activity.  For 
example,  acquisition  programs  by  definition  support  organizational 
(and  national)  strategies,  have  long-term  implications,  and,  in  general, 
help  create  the  future.  Major  acquisitions,  in  addition,  are  key  to  orga¬ 
nizational  (and  national)  success,  employ  significant  resources,  and 
command  top-level  oversight. 
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Strategic  planning  and  management  is  key  to  program  acquisition 
success,  both  in  terms  of  program  success  and  the  success  of  manage¬ 
ment  organizations.  All  defense  acquisition  programs  and  organizations 
must  succeed  in  a  dynamic  environment,  with  constantly  changing 
requirements,  priorities,  resources,  and  other  challenges  (Schwartz, 
2004).  This  dynamism  is  the  factor  that  impels  the  community  to  apply 
the  best  strategic  management  practices. 

For  these  reasons,  we  must  apply  the  best  strategic  management 
tools  and  processes  to  defense  systems  acquisition  activities.  Along  with 
other  management  tools  and  processes,  SP&M  should  be  done  well  for 
optimum  defense  acquisition  outcomes. 

Methodology 

To  better  understand  what  practices  are  succeeding  in  this  com¬ 
munity,  the  Defense  Acquisition  University  (DAU)  invited  over  3,000 
defense  systems  acquisition  personnel  who  had  attended  DAU  West 
Region  300-level  acquisition  courses  in  Fiscal  Years  2008-2011,  to 
respond  to  an  online  survey.  These  more  experienced  acquisition  profes¬ 
sionals  were  likely  to  have  been  exposed  to  the  concepts  outlined  in  the 
research.  A  broad  cross  section  of  acquisition  personnel  with  experience 
and  strategic  management  expertise  were  queried  for  both  qualitative 
and  Likert-like  quantitative  responses.  The  e-mail  invitation  explained 
the  researchers  were  interested  in  pulsing  professionals  with  SP&M 
experience. 

Responses  were  received  from  412  survey  respondents  who  rep¬ 
resented  a  wide  range  of  Army,  Navy,  Air  Force,  and  other  Defense 
Department  programs  and  acquisition  organizations.  Approximately  a 
third  of  the  survey  respondents  had  more  than  15  years’  experience  in 
acquisition  management,  with  significant  experience  using  strategic 
planning  and/or  strategic  management  methodologies.  Responses  from 
participants  who  indicated  no  strategic  planning  or  strategic  manage¬ 
ment  experience  were  removed  from  the  survey  response  data  analysis, 
leaving  295  qualified  responses  from  the  population  of  interest. 

After  identifying  the  respondents’  organization,  program,  position, 
certification  level,  and  experience  with  SP&M,  the  survey  assessed 
the  perceived  usefulness  of  a  wide  range  of  common  tools  used  for 
SP&M.  Both  roadblocks  and  facilitating  factors  for  effective  SP&M 
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were  identified,  as  well  as  the  types  of  resources  needed  for  effective 
SP&M.  Finally,  the  survey  assessed  the  overall  perceived  value  of  SP&M 
in  defense  systems  acquisition,  as  well  as  specific  organizational  and 
program  benefits. 

Since  the  intent  of  the  survey  was  to  understand  the  use  of  strategic 
planning  and  management  methodologies  in  the  Defense  Department, 
no  private  sector  inputs  were  solicited  or  received. 

About  24  percent  of  the  respondents  currently  hold  program  man¬ 
ager  (PM)  or  deputy  PM  positions.  Another  23  percent  hold  positions  as 
functional  leads.  The  remainder  comes  from  a  wide  array  of  program 
office  positions. 

Respondents  also  represent  a  wide  cross  section  of  functional  areas, 
although  the  largest  group  (33  percent)  is  in  program  management.  Other 
well-represented  functional  areas  were  life  cycle  logistics  (16  percent) 
and  systems  engineering  (16  percent).  Each  of  the  other  functional  areas 
comprised  less  than  10  percent  of  the  respondents. 

Roughly  50  percent  of  the  respondents  were  certified  at  Defense 
Acquisition  Workforce  Improvement  Act  Level  III,  with  Levels  I  and  II 
represented  by  about  a  quarter  of  the  respondents  each. 
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Findings 

Somewhat  surprisingly,  many  (approximately  one-third)  of  the  DoD 
acquisition  professionals  who  participated  in  this  research  project  have 
private-sector  experience  using  strategic  planning  and  management. 
Their  responses  highlight  many  ways  in  which  strategic  planning  and 
management  considerations  in  the  DoD  are  both  similar  and  different 
from  those  in  the  other  sectors. 

Similarities  include  the  observations  that  there  is  often  a  wide  gap 
in  understanding  of  strategic  factors  between  top  and  working  lev¬ 
els,  coupled  with  micromanagement  and  multilevel  approvals  in  both 
defense  and  nondefense  organizations.  In  both  types  of  organizations, 
participants  must  comply  with  specific  guidance  from  others,  and  deci¬ 
sions  often  involve  big  dollars,  long  timelines,  and  complex  programs. 
Survey  respondents  also  identified  that  in  both  types  of  organizations, 
leadership  shortcomings  and  inexperience  can  impede  effective  strategic 
planning  and  management;  and  that  it  is  not  uncommon  to  encounter 
many  uncertain,  contradictory,  and  frequently  changing  factors,  includ¬ 
ing  funding,  policies,  priorities,  requirements,  and  threats. 

On  the  other  hand,  defense  acquisition  managers  and  leaders  face 
some  fairly  unique  challenges.  Being  responsible  to  taxpayers  is  different 
from  being  answerable  to  shareholders,  especially  since  the  purpose  of 
defense  acquisition  activities  is  national  security,  not  profit-  or  market- 
driven  considerations. 

In  fact,  respondents  noted  that  sometimes  performance  must  be 
achieved  at  all  costs,  and  some  situations  can  have  life-or-death  impli¬ 
cations,  including  the  use  of  nuclear  weapons.  Defense  acquisition  is 
influenced  by  national  politics  and  must  comply  with  unique  federal 
regulations,  policies,  and  processes,  which  involve  requirements,  bud¬ 
geting  and  funding,  acquisition/procurement  methods,  and  personnel 
management  issues,  including  drawdown. 

Strategic  planning  is  widely  practiced  in  the  defense  acquisition 
community.  About  70  percent  of  the  survey  respondents  reported  that 
their  organization  has  a  current  strategic  plan  (although  about  20  per¬ 
cent  weren’t  sure). 
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Does  Your  Organization  Have  a  Strategic  Plan? 

Of  the  70  percent  of  respondents  with  current  strategic  plans,  about 
90  percent  use  their  strategic  plan  for  either  organizational  improvement 
(27  percent),  program  management  (19  percent),  or  both  (44  percent). 
When  asked  a  broader  question  about  the  use  of  strategic  planning  and/ 
or  strategic  management  methodologies  in  general,  only  16  percent  indi¬ 
cated  its  use  for  organizational  improvement,  whereas  over  25  percent 
use  these  methods  for  program  management.  Moreover,  the  use  of  these 
methods  for  both  organizational  improvement  and  program  management 
grew  to  47  percent  of  the  survey  respondents. 

These  results  indicate  that  although  strategic  planning/manage¬ 
ment  is  commonly  used  to  guide  organizational  development,  its  frequent 
use  for  program  management  suggests  that  this  is  a  potentially  fruitful 
area  in  which  to  seek  opportunities  for  improvement  and  cross-commu¬ 
nity  sharing  of  best  practices. 

While  these  results  indicate  that  strategic  planning  and  strategic 
management  methods  are  being  widely  applied  by  the  respondents, 
and,  by  inference,  across  our  community,  it’s  also  useful  to  understand 
the  respondents’  satisfaction  with  the  use  of  these  practices.  Although 
virtually  all  respondents  indicated  that  they  found  some  value  in  use  of 
SP&M  methodologies,  about  half  of  them  indicated  they  highly  value 
these  methods  for  improving  program  outcomes  (a  Likert  score  of  6  or 
7  on  a  1-7  scale). 

FIGURE  1.  DOES  YOUR  ORGANIZATION  HAVE  A  STRATEGIC  PLAN? 


269 


Defense  ARJ,  October  2013,  Vol.  20  No.  3:261-282 


Strategic  Planning  and  Management  in  Defense  System  Acquisition 


Overall,  How  Would  You  Rate  Strategic  Planning 
and  Management  Methodologies  in  Improving  Program 
Management  Outcomes? 

The  nature  of  the  value  provided  by  use  of  SP&M  methods  is  quite 
broad  in  this  community.  The  most  commonly  identified  benefit, 
expressed  by  a  full  85  percent  of  the  respondents,  was  better  commu¬ 
nications.  Closely  following  that  were  increased  internal  efficiencies 
(76  percent),  organizational  performance  gains  (69  percent),  major 
changes  to  business  practices  (45  percent),  and  increased  external  effi¬ 
ciencies  (37  percent). 


FIGURE  2.  OVERALL,  HOW  WOULD  YOU  RATE  STRATEGIC 
PLANNING  AND  MANAGEMENT  METHODOLOGIES  IN  IMPROVING 
PROGRAM  MANAGEMENT  OUTCOMES? 


What  Specific  Benefits  and  Outcomes  are  Associated 
with  SP&M  in  Defense  Systems  Acquisition? 

For  the  specific  respondents  who  gave  the  highest  ratings  to  the 
usefulness  of  SP&M  in  improving  program  outcomes,  increased  internal 
efficiency  and  better  communications  were  the  most  often  cited  benefit, 
followed  by  gains  in  organizational  performance. 


FIGURE  3.  WHAT  SPECIFIC  BENEFITS  AND  OUTCOMES  ARE 


ASSOCIATED  WITH  SP&M  IN  DEFENSE  SYSTEMS  ACQUISITION? 
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Many  other  specific  benefits  were  mentioned,  verbatim: 

•  More  efficient  execution  of  funds; 

•  Helps  solidify  the  resources  toward  a  common  goal  and 
priority  taskings; 

•  Improved  personnel  morale; 

•  Conserves  resources  by  industry  and  government  working 
together; 

•  Team  effectiveness 

•  Better  links  to  future  requirements  for  Program  Objective 
Memorandum  and  resource  planning; 

•  More  knowledgeable  workforce; 

•  Portrays  the  organization’s  strategic  contributions  to 
national  defense; 

•  Reduces  waste,  lack  of  focus,  and  duplication  of  effort; 

•  Gives  vector  in  highly  distributed  organizations; 

•  Increases  focus  on  the  important  vice  the  urgent; 

•  Direct,  measurable  bottom  line  results; 

•  Collaboration  and  coordination  with  other  program  man¬ 
agement  activities; 

•  Leadership; 

•  Provides  a  good  roadmap;  and 

•  Prepares  agencies  during  Base  Realignment  and  Closure 
activities. 
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The  respondents  identified  a  wide  range  of  factors  that  facilitate 
effective  implementation  of  strategic  plans,  including  leadership;  stake¬ 
holder/participant  involvement;  a  common  understanding  of  the  vision, 
mission,  strategic  intent  and  strategy,  based  on  clear,  unifying  goals  and 
objectives;  and,  of  course,  effective  communications.  In  the  latter  cat¬ 
egory,  specific  aspects  of  communications  that  were  mentioned  include 
documented  requirements;  clear  priorities,  issues,  and  plans;  listening 
to  everyone’s  ideas;  leadership’s  articulation  of  employees’  contribution; 
and  a  clear  format  for  published  products. 

A  large  number  of  leadership  factors  were  cited,  including: 

•  Vision 

•  Follow  through 

•  Commitment 

•  Resources 

•  Involvement/interest 

•  Buy-in 

•  Understanding 

•  Communication 

•  Implementation 

•  Attitude 

This  last  item  captured  a  variety  of  comments  such  as  the  assertion 
that  implementation  should  not  be  just  a  “check-the-box”  effort;  that  PMs 
should  take  a  long-term  approach,  not  day-to-day  churn;  that  manage¬ 
ment  should  play  a  part  in  the  development  of  strategic  planning  so  that 
they  will  understand  their  roles,  their  employees’  roles,  and  the  impor¬ 
tance  of  execution;  and  that  pressure  should  be  exerted  from  above  to  use 
the  tools  available  to  effectively  implement  these  processes. 
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All  the  factors  just  mentioned  aren’t  surprising,  and  are  consistent 
with  well-understood  best  practices  in  applying  SP&M  methodologies. 
However,  a  number  of  other  factors  were  raised  that  also  merit  further 
consideration.  These  include  (in  no  particular  order): 

•  Ensuring  proper  training  and  leadership  classes  to  retain  a 
knowledgeable  workforce; 

•  Positioning  a  full-time  facilitator/in-house  expert; 

•  Instilling  continuity,  including  having  a  living  document; 
persistency  (“don’t  change  halfway  through”),  transition 
into  sustainment,  continuity  through  leadership  changes, 
and  maintaining  consistent  direction; 

•  Having  a  good  governance  structure  (objective  owners, 
quarterly  reviews); 

•  Ensuring  appropriate,  stable  resources,  including  time  to 
commit  to  planning  and  implementation,  and  funding; 

•  Taking  the  time  to  do  a  good  job  (“When  done  properly, 
strategic  plans  can  be  very  effective,  but  most  managers/ 
leaders  get  impatient”); 

•  Showing  direct  impact  to  participants,  with  incentives; 

•  Encouraging  effective  teamwork; 

•  Paying  attention  to  cultural  change,  including  frequent  use 
of  SP&M  and  constant  monitoring  and  follow-up; 

•  Cultivating  a  practical  attitude,  including  open  mindedness 
and  willingness  to  face  the  brutal  facts; 

•  Seeking  perspective,  including  understanding  the  value 
streams  of  the  organization’s  products  and  services,  and 
the  global  impacts,  political  climate,  and  funding  associ¬ 
ated  with  the  effort; 

•  Establishing  executable  processes  up  front; 
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•  Having  a  plan  above  you  to  lash  up  to; 

•  Integrating  with  other  project  management  disciplines; 

•  Assessing  direct  measurable  impact  to  the  organization; 
and 

•  Making  your  customer  successful. 

Next,  participants  were  asked  to  rate  a  number  of  commonly  used 
SP&M  tools  and  methodologies,  and  to  identify  other  tools  that  they  have 
found  useful  for  SP&M.  From  this  survey,  the  most  useful  tools  (and  the 
primary  use  to  which  they  were  put)  were: 

•  Action  plans,  used  to  establish  priorities  and  clarify 
expectations; 

•  Root  cause  analyses,  used  to  establish  priorities  and  lower 
cost; 

•  Mission/strategy  mapping,  used  to  align  the  organization; 

•  Brainstorming; 

•  Program  analysis/assessment,  used  to  establish  priorities; 

•  Needs  assessment,  also  used  to  establish  priorities; 

•  Strengths,  Weaknesses,  Opportunities,  and  Threats 
(SWOT)  analyses,  used  to  establish  priorities; 

•  Stakeholder  interviews,  used  to  clarify  expectations;  and 

•  Vision  statements,  used  to  clarify  expectations  and  align 
the  organization. 

These  “most  useful”  tools  were  highly  rated  (Likert  6  or  7  on  a  1-7 
scale)  in  over  50  percent  of  the  responses. 
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For  the  respondents  who  gave  the  highest  ratings  to  the  usefulness 
of  SP&M  in  improving  program  outcomes,  the  highest  rated  tools  were: 

•  Program  analysis/assessment,  used  to  establish  priorities 
and  improve  alignment; 

•  Needs  assessment,  also  used  to  establish  priorities  and 
clarify  expectations; 

•  Mission/strategy  mapping,  used  to  align  the  organization 
and  establish  priorities; 

•  SWOT  analyses,  used  to  establish  priorities  and  clarify 
expectations;  and 

•  Action  plans,  also  used  to  establish  priorities  and  clarify 
expectations. 

Interestingly,  some  of  the  least  useful  tools  were  company  propri¬ 
etary  software,  force  field  analysis,  and  environmental  scans.  This  last 
factor  is  somewhat  confusing  since  reviews  of  the  regulatory  environ¬ 
ment  and  reviews  of  industry  trends  (which  would  be  included  in  an 
environmental  scan)  were  more  widely  used.  Perhaps  the  term  “environ¬ 
mental  scan”  wasn’t  familiar  to  respondents.  Further  discussion  with 
the  community  may  clarify  this  ambiguity. 

Other  tools  that  were  rated,  but  which  fell  somewhere  in  the  middle  on 
the  usefulness  reports,  included  scenario  planning,  Balanced  Scorecard, 
use  of  process  consultants,  and  use  of  industry  experts/futurists. 

Respondents  also  mentioned  a  wide  range  of  other  specific  tools 
that  they  are  using  to  facilitate  SP&M  in  their  organization  (Table  1). 
These  responses  are  listed  in  no  particular  order  or  grouping.  DAU  plans 
to  further  investigate  these  tools  to  understand  which  would  be  most 
appropriate  to  incorporate  in  structured  SP&M  training  for  wider  use 
across  the  defense  acquisition  community. 
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TABLE  1.  SP&M  TOOLS  BEING  USED  BY  THE  DEFENSE 
ACQUISITION  COMMUNITY 


Business  intelligence 
Dashboards 
Well-prepared  offsites 
Discovery-Driven  Plan/ 
Discovery-Driven  Growth 
Army  Strategic 
Management  System 
Probability  of  Program 
Success (PoPS) 

Systems2Win,  including 
LEAN  and  Six  Sigma  tools 
Continuous  Process 
Improvement 

Objective  risk-based  threat/ 
issue  assessments 
Analysis  of  Alternatives 
tool  (PMT  350) 

7-  or  9-Step  Standardize- 
Do-Check-Act  (SDCA) 

X-matrix 
Winsight/Project 
QuickScore  (Spyder  Strategies) 
Capability  Maturity  Model 
Integration  (CMMI) 

Quality  Function  Deployment 
Design  of  Experiments 
Bottom-to-top  communication 
Logistics  elements  review 
and  development 
Hoshin  planning 
Risk  Analysis 

Issue-  or  Action  Item-based 
program  management  software 


•  Project  Management  tools, 
practices,  and  processes 

•  Contract  negotiation  consultants 

•  8-Step  Problem  Solving 

•  Define,  Measure,  Analyze, 
Improve  Control  (DMAIC) 

•  Campaign  Planning  Process 

•  Plan  of  Action  and 
Milestones  (POAM) 

•  Assumption/strategic  risk 
analysis  (integrated  with  other 
project  management  disciplines) 

•  Objective  assessment  of 
value-added  for  various  DoD 
acquisition  processes 

•  Organizational  climate  survey 

•  Prerequisite  Trees 

•  Conflict  Diagrams 

•  Root-Cause  analysis 

•  Voice-of-the-Customer  feedback 

•  Business  Case  Analysis 

•  Theory  of  Constraints 

•  Integrated  Computer  Aided 
Manufacturing  Definition  for 
Function  Modeling  (IDEFO) 

•  Current  Reality  Trees/Maps 

•  Future  Reality  Trees/Maps 

•  Injection  Maps 

•  Competency-based 
toolsets/planning  processes 
(e.g.,  Lominger) 
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When  survey  participants  were  asked  to  identify  the  biggest  road¬ 
blocks  to  effective  SP&M,  the  top  three  were  the  lack  of  time,  lack  of 
management  commitment,  and  lack  of  follow-up.  Less  pressing,  but 
still  notable  roadblocks  included  lack  of  expertise,  lack  of  funds,  lack  of 
training,  and  ineffective  tools. 

For  the  respondents  who  gave  the  highest  ratings  to  the  usefulness  of 
SP&M  in  improving  program  outcomes,  the  lack  of  management  commit¬ 
ment  was  the  roadblock  most  often  cited,  followed  closely  by  lack  of  time. 

The  respondents  were  also  given  the  opportunity  to  identify  other 
roadblocks  to  effective  SP&M  they  have  encountered,  which  are  listed 
in  Table  2  in  no  particular  order  or  grouping. 
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TABLE  2.  ROADBLOCKS  TO  EFFECTIVE  SP&M 


Ineffective  metrics 
Senior  leadership  (PEO/PM) 
disagreement  about  strategy 
Lack  of  stakeholder/employee 
buy-in 

Lack  of  business  and 
organizational  management 
background  and  experience 
Lack  of  senior-level  vision  to 
require  strategic  planning 
Getting  commitment  from 
assigned  personnel 
Lack  of  personnel  trained  in 
acquisition  disciplines 
Poor  communication 
Lack  of  cohesive  vision 
Command  attitude  (crisis 
management,  don’t  make  waves, 
not  invented  here) 

Culture  of  "zero  mistakes” 
Management  distraction 
Unpredictable/erratic 
Congressional  budgetary 
direction 

Contracting  timelines 
Use  of  inappropriate  models 
(e.g.,  aircraft  in  space 
acquisition) 


•  Jaded  members  of  the 
organization  (regarding 
strategic  plans) 

•  Unrealistic  timelines 

•  Working  outside  of 
“requirements” 

•  Mid-management  reluctance 
to  change  (not  "real  work”) 

•  Personnel  commitment 
degradation  due  to 
Congressional  attitudes;  low 
morale 

•  Too  many  inefficient  legacy 
processes 

•  Unforeseen  external  drivers 
that  derail  plans 

•  Inability  to  match  time,  expertise, 
and  funds 

•  Inadequate  internal  controls 

•  Overwhelming  burden  of 
oversight  and  reporting 

•  Difficulty  in  tracking  strategic 
improvement 

•  Competition  among 
organizations  to 
“be  the  solution” 

•  Constant  reorganization  (Navy) 


Usefulness  of  specific  resources  for  effective  SP&M  was  also  mea¬ 
sured.  The  most  useful  resources  were  internal  staff  and  the  respondents’ 
own  personal  research  into  SP&M;  funding,  communities  of  practice, 
and  tool  experts  were  also  found  to  be  somewhat  useful.  Least  useful 
were  external  process  consultants  and  external  meeting  facilitators. 
However,  even  for  these  less  useful  resources,  about  20  percent  of  the 
responses  indicated  that  they  were  very  useful  (Likert  6  or  7  on  a  1-7 
scale).  In  short,  all  these  resources  can  be  important  for  effective  SP&M. 
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For  the  respondents  who  gave  the  highest  ratings  to  the  usefulness 
of  SP&M  in  improving  program  outcomes,  the  use  of  internal  staff  was 
cited  much  more  often  than  the  use  of  external  help.  This  would  seem  to 
indicate  that  training  our  organic  resources  to  conduct  effective  strategic 
planning  and  management  would  likely  have  more  impact  than  relying 
on  external  consultants. 


SP&M  can  be  highly  valuable  to  Department 
of  Defense  systems  acquisition  programs  and 
organizations  when  employed  by  experienced 
practitioners  and  managers. 


In  this  context,  it  is  interesting  that  the  great  majority  (69  percent) 
of  respondents  indicated  that  they  plan  to  use  SP&M  tools  and  meth¬ 
odologies  in  the  future,  although  two -thirds  of  this  community  have 
either  no  resources  committed  or  are  unaware  of  resources  committed 
for  future  SP&M. 

The  survey  also  identified  significant  interest  in  additional  training 
and  education  of  SP&M  topics.  Two-thirds  of  the  responders  indicated 
interest  in  additional  training  in  SP&M  tools  and  processes,  and  over 
75  percent  would  like  to  learn  more  about  best  SP&M  practices  for 
defense  acquisition  organizations  and  programs.  The  most  often  cited 
tools  for  which  additional  SP&M  training  was  recommended  were: 

•  Program  analysis/assessment 

•  Needs  assessment 

•  Mission/strategy  mapping 

•  SWOT  analyses 

•  Root  cause  analyses 

•  Balanced  scorecard 

•  Stakeholder  interviews 
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Summary 

The  data  collected  in  this  analysis  indicate  that  SP&M  can  be  highly 
valuable  to  Department  of  Defense  systems  acquisition  programs  and 
organizations  when  employed  by  experienced  practitioners  and  man¬ 
agers.  Moreover,  specific  high-value  activities,  tools,  processes,  and 
practices  have  been  identified,  as  have  common  roadblocks  to  effective 
SP&M.  Clearly,  the  data  reflect  that  for  SP&M  methods  to  be  success¬ 
ful,  acquisition  organization  leaders  must  understand  the  importance 
of  their  use.  Moreover,  tool  use  and  process  training  are  needed  widely 
within  the  community,  especially  for  senior  leaders.  From  these  observa¬ 
tions,  a  conclusion  can  be  drawn  that  implementation  assistance  can  be 
very  useful  and  should  have  significant  payoff  in  terms  of  organizational 
effectiveness  and  program  management  success. 
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Improving  Program  Success 
Through  Systems  Engineering 
Tools  in  Pre-Milestone  B 
Acquisition  Phase 


\‘  Daniel  Deitz,  Timothy  J.  Eveleigh, 
Thomas  H.  Hoizer,  and  Shahryar  Sarkani 


Today,  programs  are  required  to  do  more  with  less. 
With  70  percent  of  a  system’s  life-cycle  cost  set  at  pre- 
Milestone  B,  the  most  significant  cost  savings  potential  is 
prior  to  Milestone  B.  Pre-Milestone  B  efforts  are  usually 
reduced  to  meet  tight  program  schedules.  This  article 
proposes  a  new  Systems  Engineering  Concept  Tool 
and  Method  (SECTM)  that  uses  genetic  algorithms  to 
quickly  identify  optimal  solutions.  Both  are  applied  to 
unmanned  undersea  vehicle  design  to  show  process 
feasibility.  The  method  increases  the  number  of  alter¬ 
natives  assessed,  considers  technology  maturity  risk, 
and  incorporates  systems  engineering  cost  into  the 
Analysis  of  Alternatives  process.  While  not  validated, 
the  SECTM  would  enhance  the  likelihood  of  success  for 
sufficiently  resourced  programs. 
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This  article  examines  the  importance  of  developing  a  robust  Analysis 
of  Alternatives  (AoA)  early  in  the  concept  phase  of  the  acquisition  pro¬ 
gram  (prior  to  Milestone  B)  and  the  effects  such  development  may  have 
on  program  success.  While  current  statutes  require  that  program  man¬ 
agers  complete  an  AoA  for  all  Acquisition  Category  (ACAT)  programs, 
the  quality  of  the  AoA  is  the  predominant  indicator  for  program  suc¬ 
cess  and  consists  of  more  than  just  completing  a  study  (Government 
Accountability  Office  [GAO],  2009a).  In  2008,  the  Department  of  Defense 
(DoD)  had  96  major  defense  acquisition  programs,  which  experienced  a 
cost  growth  of  $296  billion  and  an  average  schedule  delay  of  22  months 
(GAO,  2009a).  The  GAO  completed  a  study  in  2009  where  it  identified 
one  of  the  key  causes  for  this  cost  and  schedule  growth  as  the  mismatch 
between  the  requirements  of  the  systems  and  the  resources  to  provide 
them  (GAO,  2009a).  GAO  further  stated  that  programs  enter  the  acquisi¬ 
tion  process  with  requirements  that  are  not  fully  understood,  cost  and 
schedule  estimates  that  are  based  on  optimistic  assumptions,  and  a  lack 
of  sufficient  knowledge  about  technology,  design,  and  manufacturing. 


While  SECTM  cannot  be  validated  until 
implemented  by  acquisition  programs,  it  is 
expected  to  increase  the  likelihood  of  successful 
programs  that,  if  sufficiently  resourced,  can 
deliver  on  time  and  on  budget. 


The  DoD  has  a  history  of  rushing  programs  into  development  or 
production  that  are  not  ready  due  to  various  program  constraints.  The 
Joint  Strike  Fighter  was  intended  to  produce  an  affordable  aircraft,  but 
ended  up  being  the  most  expensive  aircraft  program  in  DoD  with  over 
$200  billion  for  3,000  aircraft.  GAO  attributed  a  major  factor  for  the 
cost  overrun  to  the  program’s  premature  entry  into  the  engineering, 
manufacturing,  and  development  phase  prior  to  the  maturation  of  criti¬ 
cal  technologies  (GAO,  2001).  The  Navy  has  entered  into  shipbuilding 
contracts  without  fully  maturing  component  technologies,  resulting 
in  a  193  percent  cost  growth  on  Littoral  Combat  Ship  1  and  a  52-month 
delay  on  Landing  Platform  Dock  17  (GAO,  2009b).  This  rush  is  not  just 
on  large  ACAT  I  programs,  but  also  on  smaller  ACAT  programs  (Pincus, 
2012).  The  Navy  Organic  Airborne  and  Surface  Influence  Sweep  (OASIS) 
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program  just  experienced  a  cost  increase  from  $55  million  to  $135  mil¬ 
lion,  with  an  8-year  delay  in  fielding.  This  system  still  has  not  met  the 
requirement  to  continue  operating  after  being  hit  by  a  shock  wave  from 
a  mine  or  ordnance  explosion  (Pincus,  2012).  The  latest  results  from  the 
last  Department  of  Defense  Inspector  General  (DoDIG)  study  indicated 
OASIS  met  only  65  percent  of  its  shock  requirement  and  would  not  work 
(DoDIG,  2012). 

This  article  defines  a  new  Systems  Engineering  Concept  Tool  and 
a  five-step  system  engineering  Method  (SECTM)  that  we  developed  to 
increase  the  robustness  of  AoAs.  We  based  the  SECTM  design  on  the 
finding  from  the  GAO  (2009a)  study  that  examined  32  DoD  programs, 
and  the  impacts  that  the  quality  of  the  AoA  can  have  on  program  success. 
We  applied  SECTM  to  a  UUV  concept  design  to  show  the  feasibility  of 
implementing  the  process.  SECTM  includes  a  Systems  Engineering 
Concept  Tool  based  off  genetic  algorithms  to  quickly  explore  the  design 
solution  space.  While  SECTM  cannot  be  validated  until  implemented 
by  acquisition  programs,  it  is  expected  to  increase  the  likelihood  of  suc¬ 
cessful  programs  that,  if  sufficiently  resourced,  can  deliver  on  time  and 
on  budget. 


0  l 

0  1 
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Importance  of  Early  Information 

Within  the  increasingly  constrained  fiscal  environment  in  which  the 
DoD  must  operate,  program  life-cycle  cost  control  is  especially  important. 
All  DoD  programs,  no  matter  which  ACAT  level  is  involved,  follow  a  pro¬ 
gram  path  that  has  an  impact  on  life-cycle  costs.  Smaller  ACAT  programs 
can  streamline  or  skip  minor  steps,  but  the  overall  acquisition  process  is 
the  same.  The  Defense  Acquisition  University  has  defined  life-cycle  cost 
across  the  various  program  milestones  as  shown  in  Figure  1  (Defense 
Systems  Management  College,  1990).  Only  10  percent  of  the  program’s 
life-cycle  cost  is  invested  during  the  system’s  research  and  development 
phase  up  to  the  system’s  initial  operational  capability;  however,  this  may 
be  the  most  important  10  percent  of  the  system’s  life-cycle  cost.  As  this 
phase  commits  70  percent  of  the  program’s  life-cycle  costs,  focusing 
significant  time  and  effort  to  assure  that  all  alternatives  are  considered 
is  very  important. 


FIGURE  1.  DEFENSE  ACQUISITION  UNIVERSITY  DEFINED  LIFE- 
CYCLE  COSTS  ACROSS  VARIOUS  PROGRAM  MILESTONES 
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Current  Analysis  of  Alternatives 

In  today’s  environment,  program  managers  are  encouraged  to 
move  as  quickly  as  possible  to  meet  urgent  operational  requirements, 
replacement  schedules,  or  to  save  time.  Because  the  majority  of  the  pre- 
Milestone  B  work  is  level  of  effort,  shortening  this  effort  is  easier  than 
shortening  the  design  and  fabrication  work.  While  this  approach  may 
be  appealing  to  many  program  managers  and  requirements  officers, 
the  acquisition  efforts  leading  to  Milestone  B  set  the  foundation  for  the 
program.  The  work  in  this  phase  defines  the  acquisition  strategy  and 
life-cycle  cost. 

In  2009,  GAO  analyzed  32  major  defense  acquisition  program 
starts  since  fiscal  year  2003.  That  analysis  is  summarized  in  Table  1 
(GAO,  2009a). 


TABLE  1.  COMPARISON  OF  THE  SCOPE  OF  ALTERNATIVES 
CONSIDERED  WITH  PROGRAM  COST  AND  SCHEDULE  GROWTH 


Number  of  programs 
with  cost  or  schedule 
growth3 

Scope  of  alternatives11 

Low 

Moderate 

High 

No  AoA  conducted 

7 

0 

3 

AoA  included  broad  scope 

of  alternatives 

7 

i 

1 

AoA  included  narrow 

scope  of  alternatives 

4 

l 

8 

Source:  GAO. 

a  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement 
costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates, 
Moderate  =  10-24  percent  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Low  =  less  than 
10  percent  growth  in  development  cost  (or  procurement  costs  for  nondevelopmental 
programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12-month  delay  for  the  initial  operational 
capability  date  or  acquisition  cycle.  Moderate  =  7-  to  12-month  delay  for  the  initial 
operational  capability  date  or  acquisition  cycle,  Low  =  less  than  7-month  delay  for  the 
initial  operational  capability  date  or  acquisition  cycle. 

b  Narrow  scope  of  alternatives  =  2-5  alternative  within  one  concept;  broad  scope  of 
alternatives  =  8-26  alternatives  within  one  concept,  or  alternatives  within  multiple  concepts. 
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Of  the  32  major  DoD  acquisition  programs,  10  programs  did  not 
complete  a  formal  AoA.  For  at  least  seven  of  those,  this  may  have  been 
appropriate  since  they  were  modernization  or  evolutionary  programs. 
The  Defense  Acquisition  Guide,  which  states  that  an  AoA  should  focus  on 
the  end-state  solution,  contains  recommendations  on  a  single  develop¬ 
ment  or  evolutionary  development  path  (Defense  Acquisition  University, 
2012).  The  Milestone  Decision  Authority  can  waive  the  requirement 
for  a  new  AoA  for  incremental  or  modernization  efforts  included  in 
previous  analyses.  The  Navy  Standard  Missile  SM6  is  an  example  of  an 
evolutionary  acquisition  program  where  block  increments  were  used  to 
incrementally  reach  the  final  capability,  thereby  negating  the  necessity 
for  an  AoA.  Thirteen  major  acquisition  programs  conducted  a  narrow 
scope  AoA  where  over  60  percent  of  the  programs  experienced  significant 
cost  or  schedule  growth.  Nine  major  acquisition  programs  conducted  a 
broad  scope  AoA  where  only  one  of  these  programs  experienced  a  sig¬ 
nificant  cost  or  schedule  growth.  This  GAO  (2009a)  study  showed  that 
broader  scope  AoAs  had  less  cost  and  schedule  overruns. 


Because  the  majority  of  the  pre-Milestone  B  work  is 
level  of  effort,  shortening  this  effort  is  easier  than 
shortening  the  design  and  fabrication  work. 


Let  us  highlight  one  program’s  AoA  process.  The  Air  Force  needed 
to  replace  its  KC-135  tanker.  This  was  a  high-visibility  major  ACAT  I 
defense  program  for  the  Air  Force.  The  KC-135  provided  80  percent  of 
U.S.  air  refueling  capability  that  enabled  airpower  to  be  deployed  and 
sustained  overseas  in  a  timely  manner.  The  fleet  of  KC-135s  was  reaching 
50  years  of  age  and  becoming  increasingly  costly  to  maintain  and  oper¬ 
ate.  The  replacement  program  for  these  aircraft  was  expected  to  be  close 
to  $200  billion  (RAND,  2006),  and  6  months  were  allocated  for  the  AoA. 

The  KC-135  AoA  was  required  to  study  the  amount  of  fuel  the  air¬ 
craft  could  supply  along  with  the  times  and  locations  in  a  set  of  mission 
scenarios  (RAND,  2006).  The  AoA  met  these  criteria  through  analyzing 
four  major  aircraft  classes  and  seven  different  methods  to  procure  those 
classes.  However,  the  AoA  was  focused  on  only  one  major  objective:  life- 
cycle  cost.  The  AoA  assumed  that  all  threshold  requirements  must  be 
met,  so  no  analysis  was  conducted  to  see  if  any  single  requirement  was 
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driving  the  cost.  In  addition,  the  AoA  did  not  look  at  the  technology  risk 
of  the  program  to  predict  the  level  of  uncertainty  that  can  drive  program 
overruns  late  in  the  design. 

The  GAO  (2009a)  report  found  that  narrowing  the  AoA  scope  to 
life-cycle  cost  did  not  enable  the  identification  of  the  most  promising 
alternative,  and  reducing  the  AoA  schedule  did  not  allow  enough  time  to 
complete  a  thorough  analysis.  The  GAO  study  recommended  that  DoD 
develop  guidance  for  conducting  robust  AoAs  to  adequately  select  an 
alternative  (GAO,  2009a). 

Systems  Engineering  in  the 
Pre-Milestone  B  Acquisition  Phase 

Program  managers  and  resource  sponsors  are  under  increasing  pres¬ 
sure  to  perform  at  a  higher  level  with  less  resources.  It  appears  unlikely 
that  increasing  either  the  timeline  or  the  cost  of  conducting  an  AoA  is  an 
option.  We  propose  to  use  our  SECTM  in  the  AoA  process  to  thoroughly 
evaluate  additional  alternatives  in  the  same  AoA  timeline. 

The  systems  that  DoD  acquires  have  become  more  complicated,  and 
quantifying  the  effect  that  each  requirement  has  on  these  systems  is 
becoming  increasingly  difficult.  As  the  DoD  strives  to  adopt  more  com¬ 
mercial  practices,  it  will  need  to  adjust  its  acquisition  processes.  Unlike 
the  DoD,  the  commercial  industry  focuses  on  the  market  and  the  price 
point  to  enter  into  that  market.  Most  commercial  industry  program/ proj¬ 
ect  managers  attempt  to  find  the  best  value  for  the  customer  by  providing 
the  most  capability  for  a  set  price.  In  today’s  shrinking  defense  budget,  a 
more  commercial  strategy  may  be  needed  to  keep  the  same  force  levels 
and  capability  despite  reduced  funding.  According  to  Navy  Admiral 
Jonathan  Greenert  (Chief  of  Naval  Operations,  2012): 

We  can  no  longer  afford,  strategically  or  fiscally,  to  let  the  perfect 
be  the  enemy  of  the  good— or  the  good  enough— when  it  comes  to 
critical  war  fighting  capability,  (p.  7) 

Systems  engineering  provides  the  rigor  needed  to  handle  the  increas¬ 
ing  complexity  of  today’s  DoD  systems.  We  are  moving  from  lowest 
cost  for  a  set  threshold  performance  to  simultaneously  minimizing  or 
maximizing  multiple  objectives  like  minimizing  cost,  maximizing  per¬ 
formance,  and  minimizing  program  risk.  In  multiple-objective  analysis. 
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multiple  solutions  exist  that  are  each  optimal,  since  they  are  at  least  as 
good  as  any  other  solution  for  some  weighted  combination  of  the  multiple 
objectives.  For  that  reason,  these  solutions  are  referred  to  as  nondomi- 
nated,  as  they  each  have  no  other  solution  that  dominates  for  at  least 
one  weighted  combination  of  the  objectives.  The  set  of  all  nondominated 
solutions  is  referred  to  as  the  Pareto  front.  Figure  2  is  an  example  of  a 
Pareto  or  nondominated  solution  where  the  design  solutions  are  shown 
in  orange  and  the  optimal  solutions  form  a  line  shown  in  green  (Brown, 
2003).  In  a  Pareto  optimal  designed  system,  the  design  can  trade  off 
cost  versus  risk  to  find  an  optimal  solution.  As  parameters  are  varied  in 
one  optimal  solution,  they  create  other  optimal  solutions  if  the  solution 
improves  in  meeting  at  least  one  objective.  Therefore,  for  a  solution  to  be 
optimum,  it  can  only  decrease  cost  to  the  point  where  it  has  a  negative 
effect  on  performance  or  program  risk. 


FIGURE  2.  SYSTEMS  ENGINEERING  CONCEPT  TOOL  AND 
METHOD  (SECTM) 
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Using  multiple-objective  analysis,  we  developed  a  new  tool  and 
designed  a  method  (SECTM)  to  address  GAO’s  recommendation  that 
DoD  AoAs  need  to  investigate  a  broader  scope  of  alternatives  to  increase 
their  robustness  (GAO,  2009a).  The  SECTM  increases  breadth  of  the 
alternatives  considered,  investigates  program  risk  based  on  technology 
selections,  and  addresses  systems  engineering  complexity  in  the  cost 
estimate,  as  shown  in  Figure  3.  The  proposed  approach  assesses  the 
technologies,  defines  the  system  metrics,  provides  a  tool  to  evaluate  the 
alternatives,  and  can  provide  the  stakeholders  with  an  assessment  of 
optimal  alternatives.  The  alternatives  that  appear  to  be  within  the  avail¬ 
able  resources  could  proceed  to  the  formal  DoD  AoA  process. 

The  steps  in  our  proposed  process  are  detailed  in  the  following 
discussion. 

FIGURE  3.  EXAMPLE  OF  A  PARETO  OR  NON  DOMINATED  SOLUTION 
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Step  One— Assess  Availability  of  Current  Technology 

The  first  step  in  the  proposed  approach  is  to  assess  the  current 
technology  available  and  develop  models  of  those  key  technologies  or 
subsystems.  This  will  allow  a  wide  net  to  be  cast  for  investigating  tech¬ 
nologies,  typically  using  Technology  Readiness  Levels  (TRL)  1-9.  In 
2001,  the  Deputy  Under  Secretary  of  Defense  for  Science  and  Technology 
endorsed  the  use  of  TRL  in  new  major  acquisition  programs.  DoD 
Instruction  (DoDI)  5000.02  describes  TRLs  from  a  systems  perspective 
and  states  that  they  are  to  be  used  for  both  hardware  and  software  (DoD, 
2008).  While  to  date,  they  have  been  used  in  the  criteria  for  gate  reviews, 
they  do  not  fold  in  program  risk.  Subsystem  concept  models  should  be 
created  to  represent  system  performance,  cost,  and  risk,  and  include 
TRL  evaluation. 


The  cost  of  doing  systems  engineering  is  becoming 
a  significant  cost  factor  due  to  the  increased 
complexity  of  today’s  systems. 


Step  Two— Define  Objectives  for  Alternatives 

The  second  step,  which  can  occur  in  parallel  to  step  1,  is  the  defini¬ 
tion  of  high-level  objectives  (and  associated  metrics)  for  the  alternatives. 
For  robustness,  there  should  be  at  least  three  primary  objectives  consid¬ 
ered:  technical  performance,  cost,  and  risk  (GAO,  2001).  These  objectives 
will  be  used  to  rank  the  different  alternatives  and  provide  recommenda¬ 
tions  on  the  set  of  optimal  solutions  in  the  next  step. 

Technical  performance  and  cost  objectives  are  part  of  the  standard 
AoA  process  and  should  continue  to  be  defined.  In  addition  to  cost  and 
performance,  we  recommend  using  Technology  Maturity  Risk  as  a  new 
objective.  The  GAO  (2009a)  report  states  that  inadequate  technology 
maturity  is  a  key  factor  in  program  cost  and  schedule  overruns.  The  time 
has  come  to  explicitly  consider  Technology  Maturity  Risk  in  the  AoA  to 
increase  program  success. 

Researchers  at  the  University  of  Southern  California  Center  for 
Software  Engineering  have  proposed  an  approach  that  includes  technol¬ 
ogy  maturity  risk.  They  report  that  TRL  maturity  has  both  positive  and 
negative  aspects.  Higher,  more  mature  technologies  can  have  a  greater 
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risk  of  obsolescence  or  the  possibility  of  a  leap-ahead  technology  dur¬ 
ing  the  life  of  the  DoD  system.  Lower,  less  mature  technologies  have 
greater  development  cost  and  schedule  risk.  These  researchers  have 
proposed  a  new  Technology  Maturity  Risk  function  based  on  the  TRL 
of  a  technology,  the  maturity  of  the  technology  in  a  system,  and  the  risk 
of  obsolescence.  While  Technology  Maturity  Risk  has  been  considered 
in  the  past,  Valerdi  developed  a  new  model  that  links  the  Technology 
Maturity  Risk  to  a  programmatic  cost  (Valerdi,  Boehm,  &  Reifer,  2003). 
From  Valerdi’s  study,  which  included  efforts  of  over  40  systems  engineer¬ 
ing  experts,  this  Technology  Maturity  Risk  has  also  been  associated  with 
a  program  cost  impact.  Table  2  shows  the  rating  scale  for  Technology 
Maturity  Risk  (Valerdi  &  Kohl,  2004). 

Many  programs  underestimate  the  cost  of  the  large  systems  engi¬ 
neering  effort  required  to  develop  complex  systems.  Therefore,  in 
addition  to  the  life-cycle  cost  models  of  the  individual  systems  (aircraft, 
ship,  vehicle,  weapons,  information  technology,  etc.),  the  cost  models 
need  to  consider  the  systems  engineering  cost.  While  advocating  no 
particular  cost  modeling  tool,  the  authors  surmise  that,  to  properly 
determine  life-cycle  costs,  systems  engineering  costs  must  be  con¬ 
sidered  in  the  life-cycle  cost  calculation.  In  2003,  Valerdi  developed 
the  Constructive  Systems  Engineering  Cost  Model  (COSYSMO)  for 
the  purpose  of  estimating  the  systems  engineering  effort  needed  for 
large  complex  systems  (Valerdi,  Boehm,  &  Reifer,  2003).  His  analysis  is 
based  on  four  categories:  product,  platform,  personnel,  and  project  (with 
technology  risk  being  a  driver).  With  assistance  from  the  International 
Council  on  Systems  Engineering,  the  COSYSMO  model  has  been  vali¬ 
dated  with  industrial  partners  while  new  lessons  learned  are  continually 
incorporated  (Valerdi,  Rieff,  Roddler,  &  Wheaton,  2007).  The  cost  of 
doing  systems  engineering  is  becoming  a  significant  cost  factor  due  to 
the  increased  complexity  of  today’s  systems. 

Step  Three— Apply  a  Systems  Engineering  Concept  Tool 

The  heart  of  SECTM  is  our  Systems  Engineering  Concept  Tool. 
The  subsystems  models  from  step  1  and  the  objectives  from  step  2  feed 
our  Systems  Engineering  Concept  Tool.  We  recommend  that  a  genetic 
algorithm  solver  (see  further  discussion)  be  used  because  the  user  can 
select  the  number  of  alternatives  to  be  considered,  and  genetic  algo¬ 
rithms  provide  a  good  estimate  of  the  optimal  solution  set.  For  complex 
systems  with  10  or  more  critical  design  parameters,  the  number  of  dif¬ 
ferent  solutions  can  range  from  10  to  100  billion,  which  is  far  too  many  to 
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investigate.  A  genetic  algorithm  solver  quickly  defines  the  solution  space 
and  provides  a  near-optimal  solution  set  in  a  small  number  of  iterations 
(Deb,  2001).  A  classical  optimization  problem  would  compare  each  pos¬ 
sible  solution  pairwise,  for  which  there  may  be  10  billion  comparisons. 
Genetic  algorithms  can  provide  a  good  estimation  of  the  optimal  solu¬ 
tions  with  a  population  size  as  small  as  100  and  converge  as  quickly  as 
10  generations  from  the  results  of  the  UUV  example  described  later.  Deb 
(2001),  a  recognized  expert  in  genetic  algorithms,  states  that  genetic 
algorithms  have  tremendous  advantage  over  classical  search  techniques 
because  genetic  algorithms  move  the  entire  optimal  population  toward 
the  optimal  solutions  instead  of  a  single  solution. 


Genetic  algorithms  were  developed  to  imitate 
the  processes  that  evolve  in  living  beings,  and 
the  algorithms  allow  the  designs  to  evolve  each 
generation  to  better  meet  the  identified  objectives. 


Genetic  algorithms  were  developed  to  imitate  the  processes  that 
evolve  in  living  beings,  and  the  algorithms  allow  the  designs  to  evolve 
each  generation  to  better  meet  the  identified  objectives.  Even  if  they 
do  not  use  this  formal  methodology,  designs  typically  evolve  just  with 
trial  and  error  (Eddy  &  Lewis,  2001).  The  heart  of  genetic  algorithm 
research  began  with  Schaffer  in  the  1980s.  Now,  many  genetic  algo¬ 
rithms  are  available  that  can  be  applied  to  this  problem  due  to  prior 
research  (Schaffer  &  Grefenstette,1985;  Zitzler,  Deb,  &  Thiele,  2000; 
Horn,  1997).  A  few  common  algorithms  include  the  Vector  Evaluated 
Genetic  Algorithm  (VEGA),  Nondominated  Sorting  Genetic  Algorithm 
(NSGA  II),  and  Niched  Pareto  Genetic  Algorithm  II  (NPGA)  (Zitzler, 
Deb,  &  Thiele,  2000).  VEGA,  one  of  the  first  Pareto  genetic  algorithms 
from  the  1980s,  works  by  assigning  a  randomly  selected  single  objective 
to  each  member  of  the  population.  NPGA  was  developed  by  Horn  and 
Nafpliotis  in  1993,  and  improved  on  the  selection  process  determin¬ 
ing  the  dominance  of  randomly  selected  groups  of  individuals  in  the 
population  (Coello  Coello,  2000).  NSGA  II  was  developed  from  work  by 
Srinivas  and  Deb  (1994)  in  2000,  and  improved  upon  the  basic  genetic 
algorithm  by  sorting  the  population  in  multiple-level  solutions,  starting 
with  the  nondominated  and  binning  them  into  levels  of  domination  until 
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all  solutions  are  binned.  The  level  of  domination  is  used  to  modify  the 
fitness  of  individuals.  This  allows  for  quick  and  computationally  efficient 
algorithms  compared  to  other  methods  (Coello  Coello,  2000),  which  is 
why  we  selected  NSGA II  for  this  research.  Genetic  algorithms  have  been 
used  to  solve  many  complex  problems,  especially  in  aircraft  and  ship¬ 
building  where  many  objectives  compete  with  each  other.  Figure  2  shows 
an  example  of  Pareto  front  trading  off  effectiveness  or  performance  ver¬ 
sus  cost.  The  feasible  region  is  the  large  number  of  possible  alternatives 
in  orange.  The  nondominated  solutions  are  the  set  of  optimal  solutions. 
For  multiple-objective  problems,  two  solutions  are  possible:  one  domi¬ 
nates  (or  is  better  than  the  other)  or  nondominated  (each  solution  is 
equally  good  as  one  another).  Defined  by  Goldberg  (1989),  a  solution  is 
considered  nondominated  when  an  objective  cannot  be  increased  with¬ 
out  reducing  the  other  objectives.  In  complex  systems,  rarely  is  there  one 
optimal  solution,  but  rather  a  set  of  optimal  solutions.  The  stakeholders 
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need  to  make  the  final  choices  between  optimal  solutions  that  best  meet 
their  needs.  The  goal  of  our  Systems  Engineering  Concept  Tool  is  to 
identify  those  optimal  solutions. 

The  Systems  Engineering  Concept  Tool  should,  at  a  minimum, 
include  three  primary  objectives:  technical  performance,  cost,  and  risk. 
Many  AoAs  today  only  consider  the  cost  to  meet  the  threshold  require¬ 
ment,  whereas  SECTM  would  allow  key  performance  parameters  to 
be  separate  objectives,  and  cost  and  risk  could  be  traded  among  those 
key  parameters. 

Step  Four— Presentation  of  Optimal  Alternatives 
to  Stakeholders 

The  fourth  step  in  the  proposed  process  shown  in  Figure  3  is  to  pres¬ 
ent  the  optimal  alternatives  developed  from  the  Systems  Engineering 
Concept  Tool  to  the  stakeholders.  Since  one  optimal  solution  is  rarely 
applicable  in  complex  systems,  an  Executive  Steering  Group  (ESG) 
should  narrow  the  set  of  optimal  solutions  to  those  that  fall  within  the 
resources  available  and  the  program  constraints.  Through  the  use  of 
tradeoffs,  SECTM  will  provide  a  set  of  optimal  solutions  that  meet  the 
metrics  defined  in  step  2.  Since  these  solutions  are  equal  mathemati¬ 
cally,  the  ESG  needs  to  identify  or  narrowthe  “best  solutions”  dependent 
on  preferences  and  experience  (Faulkenberg  &  Wiecek,  2010).  These 
narrowed  solutions  should  then  undergo  a  detailed  analysis  by  subject 
matter  experts.  The  ESG  can  also  decide  to  change  the  metrics  to  refine 
this  analysis  if  none  of  the  alternatives  are  appropriate. 

Step  Five— Detailed  Analysis  (Similar  to  DoD’s 
AoA  Process) 

The  last  step  is  to  take  the  narrowed  set  of  optimal  alternatives  and 
complete  a  detailed  analysis  of  each  alternative.  This  step  is  similar  to 
the  DoD  AoA  process,  which  uses  a  set  of  subject  matter  experts  and 
increased  fidelity  models  and  simulations  to  determine  and  subsequently 
recommend  the  best  alternatives.  The  analysis  in  step  5  will  use  high- 
fidelity  physics  models  that  are  significantly  more  detailed  than  the 
subsystem  concept  models  used  in  step  1. 
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Summary  of  the  Five  Steps 

The  developed  methodology  described  in  the  preceding  5  steps  is 
anticipated  to  increase  the  robustness  of  the  DoD  pre-Milestone  B  Phase 
AoA  by: 

•  Widening  the  solution  space  investigated  within  the  time 
and  personnel  constraints; 

•  Incorporating  the  Technology  Maturity  Risk;  and 

•  Incorporating  the  cost  to  mitigate  Technology  Maturity 
Risk  and  the  level  of  systems  engineering  needed  for  com¬ 
plex  DoD  programs. 

Application  of  SECTM  to 
Unmanned  Systems  Concepts 

In  this  section  we  demonstrate  the  use  of  SECTM  on  unmanned 
systems  that  have  a  strong  appeal  in  the  DoD  environment.  Unmanned 
systems  take  the  DoD’s  most  valuable  asset— its  personnel— and  remove 
them  from  dull,  dirty,  and  dangerous  tasks.  These  systems  demonstra¬ 
bly  reduce  the  forward  deployments  of  our  military  personnel,  thereby 
increasing  the  quality  of  life  for  our  soldiers  and  their  families. 

Unmanned  aerial  vehicles  (UAV)  have  been  used  for  many  years 
with  high  success  in  the  war  against  terrorism.  Secretary  of  the  Navy 
Ray  Mabus  stated  his  priority  in  maintaining  the  competitive  edge  by 
moving  beyond  pilotless  UAVs  to  fielding  unmanned  undersea  vehicles 
(UUV),  as  well  as  surface  vehicles  (Mabus,  2010).  UUVs  will  provide  a 
new  capability  without  significant  experience  or  analysis  to  bound  AoA 
scope.  Since  UUVs  may  be  the  next  big  acquisition  of  unmanned  systems, 
they  are  a  good  test  case  for  SECTM.  The  following  discussion  reapplies 
the  steps  defined  earlier  using  an  analysis  of  UUV  designs  as  an  example. 

Step  One— Assess  Availability  of  Current  Technology 

The  first  step  in  applying  SECTM  was  to  analyze  the  UUV  sub¬ 
systems  and  to  determine  the  critical  technologies  in  each  subsystem. 
Subsystem  models  were  created  from  core  hydrodynamic  texts  and  UUV 
literature  from  Massachusetts  Institute  of  Technology  and  Southampton 
Universities  (Furlong,  McPhail,  &  Stevenson,  2007).  We  developed  a 
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basic  UUV  cost  model  using  the  Naval  Sea  Systems  Command  cost 
estimation  handbook,  and  adapting  a  submarine  cost  model  and  systems 
engineering  cost  models  (Valerdi,  Boehm,  &  Reifer,  2003). 

We  identified  critical  technologies  for  achieving  endurance  that 
were  based  on  experience  with  the  Autosub  UUV  (Furlong,  McPhail,  & 
Stevenson,  2007)  being  designed  for  an  endurance  of  5,000  meters  and 
buoyancy-driven  UUVs.  For  simplicity  of  this  example,  the  two  driving 
technologies  are  the  energy  density  of  the  primary  power  system  and  the 
hotel  load  (for  example  heating,  computing,  power  distribution).  Most 
current  UUV  systems  use  batteries,  which  are  a  high  TRL  (mature), 
but  low-energy  density.  However,  high-energy  density  power  systems 
like  fuel  cells  and  combustors  are  being  developed  and  show  promise 
at  low  TRLs.  We  completed  a  market  survey  to  look  at  the  different  bat¬ 
tery  technologies  and  their  energy  density  as  a  function  of  TRL.  Hotel 
power  was  linked  to  one  primary  technology— computer  processors.  We 
used  current  quad-core  processors  as  the  standard  processor  at  TRL  8. 
New  gaming  and  cell  core  processors  are  being  developed  that  have  the 
potential  to  reduce  the  processing  power  by  a  factor  of  four,  but  these  are 
only  at  TRL  3.  Once  again,  we  completed  a  market  survey  and  created  a 
model  to  link  processing  power  with  a  TRL. 

Step  Two— Define  Objectives  for  Alternatives 

Step  Two  defines  the  objectives  for  the  system  Pareto  analysis. 
We  used  the  2004  Navy  UUV  Master  Plan  as  a  guiding  document  to 
determine  the  UUV  design  objectives  (Department  of  the  Navy,  2004). 
The  first  objective  was  to  maximize  the  endurance  or  range  of  a  UUV  to 
be  able  to  perform  Navy  missions  like  mine  warfare  and  intelligence, 
surveillance,  and  reconnaissance.  The  second  design  objective  was  to 
minimize  the  UUV’s  volume.  This  is  important  for  integration  of  the 
UUV  onto  existing  Navy  platforms  since  larger  UUVs  may  not  fit  on 
many  Navy  ships.  The  third  and  fourth  objectives  were  cost  and  tech¬ 
nology  risk. 
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Figure  4  illustrates  the  10  different  parameters  we  considered  in  this 
UUV  analysis  and  links  those  parameters  to  each  objective. 

FIGURE  4.  SECTM  OBJECTIVES  AND  SYSTEM  PARAMETERS 

Four  design  objectives  to  optimize: 

Range  =  f  (  I,  d,  B,  V,  PH,  r\m,  np)  Maximize 
Volume  =  f  ( I,  d,  B,  PH,  D,  VM,  ct)  Minimize 
Cost  =  f  ( I,  d,  D,  B,  TRLs)  Minimize 
Technology  Risk = f  (  TRLs)  Minimize 


Design  Parameters  for  UUVs: 

1.  Length  (1) 

5.  Velocity  (V) 

9.  Propulsion  Efficiency  fry  ) 

2.  Diameter  (d) 

6.  Maximum  Velocity  (VM) 

10.  Depth  (D) 

3.  Energy  Density  (B) 

7.  Hotel  Power  (PH) 

4.  Foam  Specific  Gravity  (a) 

8.  Motor  Efficiency  (nm) 

Early  communication  with  stakeholders  on 
potential  alternatives  can  facilitate  a  better 
understanding  of  the  requirements. 


Step  Three— Apply  a  Systems  Engineering  Concept  Tool 

We  chose  the  NSGA-II  genetic  algorithm  developed  by  Deb  (Deb, 
Pratap,  Agarwal,  &  Meryarivan,  2002)  for  our  basic  genetic  algorithm 
solver  because  of  this  algorithm’s  computational  efficiency.  NSGA-II’s 
computational  efficiency  can  be  approximated  by  the  formula:  /(MV) 
as  opposed  to  other  sorting  algorithms,  which  us ef(M*N3)  where  M  is 
the  number  of  objectives  and  N  is  the  genetic  population  size.  For  the 
UUV  example  where  M=  3  and  IV  =  100,  NSGA-II  saved  2,970,000  com¬ 
putations.  We  programmed  equations  for  each  of  the  design  objectives 
into  Matrix  Laboratory,  or  MATLAB  programming  language  using  the 
NSGA-II  algorithm  for  the  optimization. 
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Since  the  population  size  is  variable,  it  can  be  selected  by  the  users. 
Increased  population  size  provides  more  points  on  the  Pareto  front  to 
better  identify  design  trends.  However,  increased  population  size  will 
square  the  number  of  computations  required. 

For  this  example,  there  are  10  basic  design  parameters  (r).  For  simpli¬ 
fication,  if  each  design  parameter  had  only  10  different  applicable  values 
( n ),  then  through  pairwise  comparison: 

Estimated  number  of  comparison  =  nr 

For  this  example,  n=  10  and  r  =  10,  which  is  10  billion  combinations 
that  would  have  to  be  analyzed.  This  is  far  too  many  to  accomplish  in  just 
a  few  months;  however,  the  use  of  a  genetic  algorithm  solver  reduces  the 
number  of  processes  significantly.  The  genetic  solver  starts  with  random 
solutions.  Those  solutions  that  have  a  higher  match  to  the  objectives  are 
selected  for  regeneration  and  combined  together  to  create  a  new  genera¬ 
tion.  This  process  is  continued  and  mimics  the  way  living  species  survive 
and  adapt  to  the  environment.  Genetic  algorithms  usually  can  converge 
in  10  generations;  therefore,  the  amount  of  calculation  needed  is  the 
population  size  times  the  number  of  generations: 

Number  of  designs:  population  x  generations  =  100  x  10  =  1,000 

The  results  of  this  application  show  that  for  the  UUV  design  dis¬ 
cussed  here,  which  is  fairly  simple  compared  to  many  DoD  systems, 
the  solution  space  of  10  billion  different  design  combinations  can  be 
approximated  by  a  population  size  of  10  or  100  with  less  than  1,000 
design  iterations  using  the  SECTM.  SECTM  is  a  very  efficient  way  to 
determine  a  set  of  optimal  alternatives  to  present  to  the  stakeholders. 

Early  communication  with  stakeholders  on  potential  alternatives 
can  facilitate  a  better  understanding  of  the  requirements.  Today’s 
systems  are  so  complex  and  highly  integrated,  that  it  is  impossible  to 
understand  the  large  impacts  that  small  changes  can  make  without  the 
use  of  analysis  tools.  SECTM  provides  a  visualization  of  the  tradeoffs  of 
risk,  cost,  and  technical  performance.  These  tradeoffs  are  very  important 
to  the  success  of  the  program.  Using  current  practices,  stakeholders  are 
not  presented  with  enough  data  to  make  good  decisions.  Steps  4  and  5 
were  not  completed  in  this  example  as  they  feed  the  DoD  AoA  process 
and  were  not  needed  to  show  the  feasibility  of  the  SECTM. 
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Conclusions 

In  today’s  reduced  budget  and  constrained  fiscal  environment,  mak¬ 
ing  acquisition  decisions  that  provide  the  best  value  to  the  nation’s  armed 
forces  and  the  DoD  is  extremely  important.  Over  70  percent  of  a  system’s 
life-cycle  cost  is  determined  by  Milestone  B;  therefore,  the  largest  impact 
can  be  made  during  these  early  program  stages.  Unfortunately,  this  is 
where  a  large  majority  of  programs  streamline,  reduce,  or  cut  activities  to 
save  time  and  funding.  Out  of  32  programs  reviewed  by  the  GAO  (2009a), 
60  percent  of  the  programs  that  completed  limited  scope  in  their  AoAs 
experienced  significant  cost  and/or  schedule  overruns  compared  to  less 
than  10  percent  in  those  programs  that  completed  a  robust  AoA. 

Applying  the  SECTM  in  the  pre-Milestone  B  Acquisition  Phase  is  an 
option  to  increase  the  AoA’s  robustness  without  significantly  increasing 
cost  or  time.  Current  processes  use  a  team  of  experts  to  analyze  a  few 
predetermined  alternatives  (three  to  10  for  a  typical  acquisition  pro¬ 
gram)  and  primarily  conduct  interviews  to  make  subjective  analyses. 
This  article  proposed  a  SECTM  to  be  used  in  the  AoA  process  to  help 
determine  or  down-select  the  few  alternatives  that  are  investigated 
in  depth  by  an  AoA  team.  When  we  applied  our  SECTM  to  a  UUV,  we 
were  able  to  reduce  10  billion  design  combinations  to  a  set  of  only  a  few 
optimal  solutions.  This  initial  systems  engineering  step  can  be  done 
rapidly  using  modeling  and  simulation  tools,  and  by  using  the  engineer¬ 
ing  process  to  down-select  the  alternatives  instead  of  a  steering  group 
committee  process. 

This  article  also  presented  the  importance  of  the  pre-Milestone  B 
Acquisition  Phase  in  setting  the  foundation  for  the  success  of  the  pro¬ 
gram.  This  methodology  presented  a  way  to  increase  the  robustness  of 
the  alternatives  considered  in  pre-Milestone  B  acquisition  documenta¬ 
tion  (primarily  AoAs)  and  incorporates  the  following  new  aspects: 

•  Widening  the  solution  space  investigated  within  the  time 
and  personnel  constraints; 

•  Incorporating  the  Technology  Maturity  Risk;  and 

•  Incorporating  the  cost  to  mitigate  Technology  Maturity 
Risk  and  the  level  of  systems  engineering  for  complex  DoD 
programs. 
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While  pre-Milestone  B  efforts  only  account  for  less  than  10  percent 
of  the  total  life-cycle  cost,  they  are  the  most  important  10  percent  of 
funding  because  they  set  the  acquisition  program  on  a  sound  foundation 
and  business  case.  Errors  in  this  phase  cost  between  three  and  10  times 
more  to  fix  in  later  phases.  The  GAO  recommended  to  the  DoD  that  new 
criteria  should  be  set  for  execution  of  AoAs,  with  the  DoD  agreeing  to 
the  recommendation.  The  approach  proposed  in  this  article  is  a  way  to 
increase  the  robustness  of  DoD’s  AoAs. 
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Time  Is  Money 


Roy  L.  Wood 


Program  managers  typically  focus  on  controlling  costs 
and  delivering  a  quality  product.  The  acquisition  stool’s 
third  leg— program  schedule— appears  to  be  a  resource 
that  can  be  slipped  to  accommodate  unstable  funding 
or  technical  difficulties.  Despite  studies  linking  high 
program  cost  and  long  schedules,  few  major  defense 
acquisition  programs  are  completed  in  less  than  a 
decade.  Programs  with  longer  schedules  experience 
further  schedule  slips,  exacerbating  the  problem.  This 
article  is  based  on  research  presented  at  the  2012  Naval 
Postgraduate  School’s  9th  Annual  Research  Symposium. 
It  includes  a  review  of  the  extant  literature  on  cost  and 
schedule  relationships,  presents  analysis  of  a  survey  of 
program  manager  perceptions  and  master  schedule 
usage,  and  examines  why  schedules  may  be  problematic 
to  acquisition  success. 
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Program  success  is  traditionally  measured  by  cost,  schedule,  and 
performance.  When  issues  arise,  trade-offs  between  these  three  are 
made;  conventional  wisdom,  however,  says  the  program  manager  (PM) 
can  generally  preserve  only  two  of  the  three.  For  example,  if  current 
budgets  are  cut,  then  programs  are  forced  either  to  give  up  some  “bells 
and  whistles”  in  performance,  or  lower  the  spend  rate  and  stretch  out  the 
program  schedule.  If  the  program  schedule  is  delayed  for  reasons  such 
as  lagging  technology  readiness  or  testing  failures,  then  program  costs 
will  rise  or  the  scope  of  the  program’s  content  will  have  to  be  sacrificed. 

When  making  such  trade-offs,  reasonably  good  tools  and  techniques 
are  available  for  estimating  cost  impacts  and  performance  trades  are 
usually  understandable.  However,  when  it  comes  to  program  schedules, 
trade-offs  can  be  much  less  clear  and  the  impacts  more  difficult  to  deter¬ 
mine.  “Working  harder”  or  placing  more  “management  emphasis”  on  an 
area  are  often  viewed  as  ways  to  improve  performance  and  “compress” 
schedules  to  remain  on  track.  These  ideas  can  lead  to  an  overly  optimis¬ 
tic  attitude  that,  unlike  money,  time  is  somehow  elastic  and  forgiving. 
This  also  leads  to  a  skewed  perception  about  the  value  of  program  time 
in  the  future  versus  the  present.  Resource  problems  in  the  near  term 
are  often  “solved”  by  pushing  work  into  the  future,  moving  milestones 
forward  while  keeping  the  program  end  date  static,  while  simultane¬ 
ously  compressing  all  the  activities  in  between.  This  forces  activities  to 
become  more  concurrent  and  increases  the  complexity  of  coordinating 
and  synchronizing  program  activities. 


While  most  of  us  have  heard  the  truism  that  “ time 
is  money,”  little  evidence  has  emerged  that  PMs 
perceive  aggressively  managing  time  and  schedules 
can  help  control  costs. 


Purpose 

This  article  explores  the  relationship  between  program  time  and  cost. 
While  most  of  us  have  heard  the  truism  that  “time  is  money,”  little  evidence 
has  emerged  that  PMs  perceive  aggressively  managing  time  and  schedules 
can  help  control  costs.  As  an  exploratory  effort,  this  research  examined 
the  literature  on  program  scheduling,  reasons  program  schedules  were 
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not  adhered  to,  and  the  relationship  of  program  schedules  to  cost  and 
program  performance.  Using  a  survey  of  73  PMs  attending  senior  courses 
at  the  Defense  Acquisition  University  in  February  2012,  this  article  also 
examines  PM  uses  and  attitudes  toward  their  own  program  schedules. 

Length  of  a  Program 
Contributes  to  Program  Cost 

The  Packard  Commission  noted  in  1986  that,  “an  unreasonably  long 
acquisition  cycle  ...  is  a  central  problem  from  which  most  other  acqui¬ 
sition  problems  stem”  (p.  8).  Echoing  this  sentiment  20  years  later,  the 
Defense  Acquisition  Performance  Assessment  report  recommended 
that,  instead  of  waiting  decades  for  100  percent  performance,  programs 
be  held  to  a  “time-certain  development”  period  of  6  years  from  the  initial 
milestone  (MS-A)  to  delivery  of  a  militarily  useful  capability  (Kadish, 
2006,  p.  12).  The  report  enumerates  some  of  the  benefits  of  shorter 
development  cycles: 

•  Operators  with  a  basic  capability  in  hand  would  gain  a  bet¬ 
ter  understanding  of  full  requirements  to  be  inserted  in 
future  increments. 

•  Technology  in  the  initial  design  would  be  at  a  higher  readi¬ 
ness  level,  and  would  mature  during  the  period  between  first 
deliveries  and  subsequent  increments. 

•  New  requirements  and  technologies  would  be  intention¬ 
ally  inserted  in  later  increments,  removing  the  temptation 
to  perturb  the  current  development  and  adding  stability  to 
the  acquisition. 

•  Reducing  time  in  development  would  also  help  add  funding 
stability  across  the  entire  program  portfolio  (Kadish,  2006, 
pp.  12-13). 

In  a  dissertation  study  at  the  Massachusetts  Institute  of  Technology 
of  154  defense  projects,  McNutt  (1998)  found  that  cost  increased  on  a 
4th  power  scale  with  development  time.  Figure  1  shows  a  derivation  of 
McNutt’s  “best  fit”  power  relationship  on  a  linear  plot.  One  can  observe 
that  the  “knee  in  the  curve”  where  costs  begin  to  escalate  significantly  is 
around  6.5  years,  indicating  that  costs  for  programs  with  schedules  that 
extend  beyond  this  point  risk  quickly  becoming  unaffordable. 
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FIGURE  1.  DEVELOPMENT  COST  VERSUS  TIME  (LINEAR  SCALE 
FOR  CLARITY) 


*  *  *  *  *  *  # 


More  recently,  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  issued  an  update— better  Buying  Power  2.0— 
that  recognizes,  and  proposes  to  reduce,  some  of  the  factors  that  increase 
program  development  (Kendall,  2012).  According  to  the  memorandum, 
these  include  “oversight  activities,  funding  stability,  contracting  lead 
time,  requirements  processes,  technical  complexity,  use  of  risk  reduc¬ 
tion  factors,  and  testing  requirements”  (p.  5).  Some  of  those  factors  are 
examined  in  this  article. 

Complex  Technical  Requirements 

A  number  of  reasons  may  explain  why  the  length  of  a  program 
impacts  its  cost  so  dramatically.  First,  a  longer  schedule  may  be  an 
acknowledgement  of  complex  technological  requirements  that  contribute 
to  a  program’s  developmental  challenges.  Programs  like  the  F-35  Joint 
Strike  Fighter  or  the  DDG-1000  naval  destroyer  with  substantial  capa¬ 
bility  needs,  advanced  technology,  unique  features,  or  with  significant 
integration  or  interdependencies  with  other  programs,  can  be  expected 
from  the  outset  to  take  longer  to  develop,  cost  more,  and  have  greater  risk. 

Evolving  requirements  and  technology  advancements. 

Programs  with  long  timelines  may  also  be  subject  to  more  requirements 
changes  as  threats  and  technologies  evolve  over  time.  As  new  threats 
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emerge  during  a  program’s  development,  there  would  be  a  need  to  try 
to  address  those  threats  by  adding  or  refining  the  delivered  system’s 
capabilities. 

Requirements  changes  during  a  program’s  lifetime  can  have  sub¬ 
stantial  impact  on  schedule  and  cost.  In  a  recent  Center  for  Strategic 
and  International  Studies  analysis,  $37  billion  of  cost  overruns  in  the 
major  defense  acquisition  program  portfolio  are  linked  to  schedule,  and 
the  report  notes  that  defense  contractors  cite  requirements  changes  late 
in  a  program  as  a  major  cause  of  schedule  impacts  (Berteau,  Hofbauer, 
Sanders,  &  Ben-Ari,  2010). 


Requirements  changes  during  a  program’s  lifetime 
can  have  substantial  impact  on  schedule  and  cost. 


Similarly,  a  recent  industry  white  paper  noted,  “frequent  and  ‘inside 
lead  time’  changes  to  program  requirements  and  production  schedules 
are  major  obstacles  to  successful  cost  and  schedule  attainment  for 
most  aerospace  and  defense  programs”  (Archstone  Consulting,  2012). 
This  assertion  is  backed  up  by  the  2008  Government  Accountability 
Office  (GAO)  analysis  of  defense  weapons  systems,  which  states  that 
“63  percent  of  the  programs  we  received  data  from  had  requirements 
changes  after  system  development  began.  These  programs  encountered 
cost  increases  of  72  percent,  while  costs  grew  by  11  percent  among  those 
programs  that  did  not  change  requirements”  (p.  5). 

When  requirements  changes  occur  during  development,  replanning 
and  rework  follow.  New  requirements  must  be  flowed  down  as  allocated 
functions  via  the  systems  engineering  process.  This  becomes  particu¬ 
larly  challenging  after  Critical  Design  Review  when  a  system’s  baseline 
is  approved  and  the  system  is  deemed  ready  to  proceed  to  fabrication, 
demonstration,  and  test.  Any  new  requirements  must  be  engineered 
and  integrated  as  well  as  possible  into  existing  program  plans.  This 
adds  complexity  and  takes  time  and  care.  Drawings  and  specifications 
must  be  revised,  schedules  and  task  budgets  altered,  test  plans  modi¬ 
fied,  and  resources  allocated  or  shifted  to  attend  to  new  or  modified 
tasks.  In  almost  any  conceivable  circumstance,  wasted  prior  effort  will 
be  scrapped  and  rework  will  be  required  to  accommodate  new  changes, 
exacerbating  the  delay  and  disruption  created  by  the  new  requirements. 
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Also,  given  the  relatively  few  new  program  starts,  the  temptation  is 
ever  present  for  requirements  managers  and  technologists  to  demand 
more  capabilities  in  each  new  program.  The  need  to  meet  forecasted 
future  threats  can  also  drive  an  appetite  among  users  toward  ever 
greater  technical  capabilities.  Stressing  requirements  for  the  Air  Force’s 
fifth-generation  fighter,  the  F-22  Raptor,  for  example,  included  stealthy 
titanium  and  composite  structure,  advanced  avionics,  active-array  radar, 
supersonic  cruise,  and  other  enabling  technologies.  Likewise,  the  Navy’s 
Zumwalt-class  destroyer  included  requirements  for  reduced  crew  size, 
advanced  active-array  radar,  integrated  power  system,  electric  drive, 
stealthy  hull,  integrated  superstructure,  155  mm  gun,  and  peripheral 
launching  system.  Many  of  these  technologies  were  developed  and 
matured  concurrently  during  the  program  engineering  and  manufactur¬ 
ing  development  phases  (Francis,  2005;  GAO,  1998). 

Schedule  and  cost  uncertainty  are  high  for  new  technology  devel¬ 
opment,  but  this  uncertainty  becomes  a  substantial  risk  when  overlaid 
on  program  milestones  that  depend  upon  successful  delivery  of  the 
technology  to  support  testing  and  fielding  a  new  system.  When  several 
new  technology  developments  are  ongoing,  as  in  the  case  of  the  F-22  and 
the  Zumwalt  destroyer,  this  uncertainty  multiplies,  and  orchestration 
of  technology  insertion  becomes  extremely  challenging.  It  should  be  of 
little  surprise  that  both  these  programs  delivered  substantially  late  and 
over  budget  (Bolkcom,  2009;  O’Rourke,  2012). 

Budget  churn  Unstable  funding  has  often  been  blamed  for  pro¬ 
gram  schedule  and  cost  issues.  Langbein  (2004)  cites  three  different 
types  of  funding  instability  that  impact  programs.  The  first  is  perhaps 
the  most  obvious— quantity  of  dollars.  These  are  programs  with  insuf¬ 
ficient  total  funding  to  perform  the  required  tasks  to  deliver  the  system. 
Underfunded  programs  can  be  caused  by  poor  cost  estimating  for  the 
funding  that  is  needed  by  the  program,  unforeseen  and  unbudgeted 
changes,  or  overly  optimistic  cost  targets.  In  defense  acquisition,  two 
other,  less  obvious  pitfalls— “color”  of  money  and  timing  of  money— can 
create  program  instability  and  schedule  problems. 

In  every  program,  defense  managers  must  break  down  total  fund¬ 
ing  into  its  constituent  functions  and  categories  of  funding  for  specific 
portions  of  the  program.  Research  and  development,  procurement, 
operations  and  maintenance,  shipbuilding  and  conversion,  and  other 
“colors”  of  money  must  be  appropriately  aligned  to  fund  the  associated 
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tasks  in  the  program.  At  times,  a  program  may  have  sufficient  overall 
funding,  but  an  incorrect  allocation  within  the  colors  of  money.  The  PM 
may  find  shortfalls  in  some  areas  and  surpluses  in  others,  but  not  have 
the  authority  to  move  the  Congressionally  appropriated  dollars  from 
one  account  to  another.  These  shortfalls  can  stop  activities  in  parts  of 
the  program  and  create  overall  delays.  Timing  of  funding  can  also  cre¬ 
ate  similar  problems.  Program  budgets  are  closely  aligned  with  planned 
work  in  any  given  year.  If  challenges  or  opportunities  arise  within  the 
year  of  execution,  current  year  funding  may  not  be  sufficient  to  accom¬ 
modate  new  funding  requirements,  again  creating  potential  delay  and 
disruption  to  the  program  schedule. 

In  each  of  these  cases,  program  schedules  must  be  replanned  to 
accommodate  different  funding  realities.  Reprogramming  or  repur¬ 
posing  current  year  funding  is  generally  not  simple  or  quick.  Requests 
for  more,  or  different,  funding  can  take  up  to  2  years  to  realize  through 
the  Planning,  Programming,  Budgeting,  and  Execution  system.  Tasks 
must  be  trimmed  in  the  current  year  and  work  adjusted  so  the  finan¬ 
cial  impacts  can  be  addressed  over  the  longer  term.  The  results  may  be 
that  key  events  and  milestones  are  missed,  concurrency  increases,  and 
opportunities  are  lost. 

Longer  programs  potentially  suffer  greater  budgetary  churn.  Each 
new  fiscal  year  presents  an  opportunity  for  decision  makers  outside 
the  program  to  make  funding  “adjustments”  that  perturb  the  program’s 
overall  performance.  Likewise,  longer  programs  with  large  budgets  can 
be  tempting  targets  for  comptrollers  or  Congress. 
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Developing  Schedules 

The  Scheduling  Process 

The  schedule  development  process  itself  may  be  a  culprit  in  program 
cost  overruns.  Ideally,  program  schedules  are  developed  in  a  rational 
and  linear  manner.  Program  requirements  are  analyzed  and  developed 
through  the  iterative  systems  engineering  process,  allocating  required 
functions  to  be  performed  by  the  system’s  hardware,  software,  and 
operators.  A  work  breakdown  structure  (WBS)  is  created,  assigning 
those  functions  to  subsystems  and  components.  At  the  lowest  level  of 
the  WBS,  work  packages  are  developed  that  allow  accurate  estimation 
of  the  resources— dollars,  time,  and  manpower/expertise— required  to 
build,  test,  and  deliver  the  hardware  or  software  widget.  Rolling  up  these 
detailed  resource  requirements  to  higher  levels  of  the  WBS,  then,  allows 
the  program  management  team  to  create  an  overall  program  budget  esti¬ 
mate,  resource-loaded  schedule,  and  program  manpower  estimate.  The 
resource-loaded  schedule  will  show  each  task  with  its  estimated  dura¬ 
tion  and  linkages  to  other  tasks  to  show  dependencies  (e.g.,  Task  B  cannot 
start  until  Task  A  is  complete),  major  milestones  where  tasks  culminate 
in  a  defining  program  event,  the  calculated  program  end  date,  and  associ¬ 
ated  critical  path.  If  each  task  is  then  given  the  appropriate  resources  and 
completes  within  its  estimated  timeframe,  then  the  program  execution 
proceeds  from  start  to  finish  with  nary  a  problem. 

Unfortunately,  this  is  generally  not  how  program  schedules  are  con¬ 
structed.  Project  end  dates  are  more  often  arrived  at  from  a  capability 
“need  date”  established  early  in  the  program  by  the  user  or  sponsor.  In 
the  survey  of  program  management  students  at  the  Defense  Acquisition 
University  in  2011,  only  18  percent  reported  that  their  program  end  date 
was  determined  through  a  roll-up  of  task  level  schedules,  while  58  per¬ 
cent  reported  end  dates  determined  by  need. 

Apparently,  herein  lies  the  source  of  some  inherent  program  prob¬ 
lems.  In  this  method  of  program  end  date  determination,  the  need  date  or 
end  date  is  fixed  and  the  program  milestones  are  “backed  out”  from  there. 
Other  project  tasks  are  fitted  into  the  milestone  scheme,  along  with  what¬ 
ever  concurrency  and  optimism  are  needed  to  make  the  schedule  “work.” 
Fully  82  percent  of  the  program  management  students  participating  in 
the  DAU  survey  reported  that  their  program  schedules  contained  some 
to  significant  concurrency  with  moderate  to  high  schedule  risk. 
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In  execution,  to  stay  on  schedule  programs  must  accept  risks,  and, 
as  the  GAO  has  noted  in  its  annual  Assessments  of  Selected  Weapons 
Programs ,  proceed  through  milestones  before  achieving  requisite  design 
and  engineering  maturity  (GAO,  2009,  2010,  2011).  This  results  in 
programs  being,  “at  a  higher  risk  for  cost  growth  and  schedule  delays” 
(GAO,  2011). 


Ninety-six  percent  of  those  polled  reported  that 
having  an  integrated  and  up-to-date  schedule 
is  critical  to  running  their  programs,  and 
two-thirds  express  confidence  in  the  accuracy 
of  their  master  schedules. 


Scheduling  processes  are  not  well  understood  or  executed. 

The  process  of  scheduling  itself  is  difficult  and  may  not  always  be  as 
useful  as  it  could  be  as  a  key  project  management  tool.  The  National 
Defense  Industrial  Association  (NDIA)’s  Industrial  Committee  for 
Project  Management  recognized  that  the  art  of  scheduling  for  complex 
projects  was  problematic  for  government  programs  and  chartered  the 
Program  Planning  and  Scheduling  Subcommittee  to  create  the  Planning 
and  Scheduling  Excellence  Guide  (NDIA,  2012).  This  guide  was  designed 
to  assist  government  and  industry  in  creating  more  useful,  consistent, 
and  standardized  integrated  master  schedules  (IMS)  using  the  prin¬ 
ciples  of  the  internationally  recognized  standard,  Generally  Accepted 
Scheduling  Practices.  The  guide  emphasizes  practical  skills  and  applica¬ 
tion  of  sound  scheduling  principles  to  create  a  schedule  that  models  the 
acquisition  plan,  provides  tips  for  schedule  maintenance,  and  advice  for 
project  managers  to  use  the  IMS  more  appropriately  to  manage  a  complex 
government  program. 

The  survey  of  PMs  at  the  Defense  Acquisition  University  revealed 
some  insights  into  how  schedules  are  viewed  by  current  managers. 
Ninety-six  percent  of  those  polled  reported  that  having  an  integrated  and 
up-to-date  schedule  is  critical  to  running  their  programs,  and  two-thirds 
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express  confidence  in  the  accuracy  of  their  master  schedules.  However, 
less  than  half  reported  that  their  schedule  is  accurately  resource-loaded, 
and  only  51  percent  are  confident  their  schedule  includes  all  the  work 
required  to  be  done  by  government  and  contractors.  These  results  seem 
to  be  inconsistent  and  perhaps  contradictory.  Only  half  responded  that 
their  schedules  are  complete  and  accurate,  yet  most  have  confidence  in 
the  schedule  and  overwhelmingly  affirm  its  importance  to  their  pro¬ 
gram’s  success!  The  Table  shown  here  summarizes  these  views. 


TABLE.  PROGRAM  MANAGER  VIEWS  OF  THEIR  SCHEDULES 


Statement 

Agree  or 
Strongly 
Agree 

Neutral, 
Disagree, 
or  Strongly 
Disagree 

Having  an  integrated  and  up-to-date 
schedule  is  critical  to  running  my 

program. 

96% 

4% 

1  have  confidence  in  the  accuracy  of 
my  master  schedule. 

65% 

35% 

My  schedule  is  accurately  resource- 

loaded. 

45% 

55% 

My  program  schedule  is  realistic  and 

achievable. 

56% 

44% 

My  schedule  includes  all  required 
work,  including  that  of  government 
organizations,  all  contractors,  and 

subcontractors. 

51% 

49% 

Similarly,  in  execution,  56  percent  responded  that  their  schedule  is 
realistic  and  achievable,  while  40  percent  report  that  their  programs  are 
behind  schedule.  When  faced  with  hypothetical  budget  cuts,  48  percent 
indicated  they  would  defer  requirements  or  capabilities,  while  only  20 
percent  would  slip  schedule  as  a  preferred  method  to  manage  overruns. 
However,  the  PMs  assigned  the  highest  priority  for  their  programs  as 
ensuring  quality  and  performance  of  their  products,  and  they  ranked 
controlling  program  scope  last  in  relative  priority.  Again,  while  their 
responses  to  questions  of  importance  align  closely  with  current  policies 
of  adjusting  scope  to  budget,  the  practical  priorities  on  performance  and 
scope,  as  reported  by  the  PMs,  would  seem  contradictory. 
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Finally,  when  the  statement  is  posed,  "Maintaining  an  accurate 
detailed  schedule  is  too  labor-intensive  and  costly  for  the  value,”  fewer 
than  10  percent  agreed  or  strongly  agreed.  When  asked  about  current  pro¬ 
gram  issues,  respondents  reported  difficulty  in  synchronizing  schedules 
among  players  second  only  to  unstable  funding.  Again,  this  seems  to  indi¬ 
cate  the  importance  PMs  place  on  the  theoretical  value  of  their  IMS,  and 
a  recognition  that  large  program  scheduling  is,  in  practice,  challenging. 


Program  problems  can  be  created  when  program 
teams  and  stakeholders  underestimate  the 
challenges  and  overestimate  their  abilities  to 
deliver  on  “success-oriented,”  aggressive  schedules. 


Overoptimism  can  lengthen  program  schedules  and  increase 
costs.  Given  that  many  program  end  dates  are  set  well  before  any  of 
the  work  begins,  perceived  necessity,  concurrency,  and  optimism  drive 
milestone  schedules  and  tasks.  Once  the  analysis  of  the  work  that  must 
be  accomplished  is  underway,  tremendous  pressure  is  pervasive  in  keep¬ 
ing  to  original  agreements  and  promises  to  deliver,  however  unrealistic. 
In  the  “Conspiracy  of  Optimism”  white  paper,  the  International  Centre 
for  Complex  Project  Management  (ICCPM,  2010)  authors  explain: 

Once  initial  project  budgets  and  schedules  are  set,  based  on  such 
estimates,  they  have  immense  staying  power,  driven  by  collective 
unrealistic  expectations,  even  to  the  extent  that  over  time,  sys¬ 
tem  functionality  and  project  resources  are  sacrificed  in  order 
to  achieve  what  was  unobtainable  in  the  first  place. 

Program  problems  can  be  created  when  program  teams  and  stake¬ 
holders  underestimate  the  challenges  and  overestimate  their  abilities 
to  deliver  on  “success-oriented,”  aggressive  schedules.  This  optimistic 
thinking  necessarily  flows  down  from  the  government  to  the  system 
contractors.  Once  the  government  team  has  fallen  into  its  own  overly 
optimistic  decision  trap,  contract  requests  for  proposal  are  written  based 
on  the  ill-conceived  plan,  and  contractors  then  come  to  the  table  hoping 
to  have  the  most  attractive  bid  to  meet  the  government’s  ill-conceived 
expectations.  Unfortunately,  this  also  often  creates  an  environment 
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where  realistic  assessments  of  cost  and  schedule  are  undervalued,  and 
contractors  who  refuse  to  join  in  on  the  conspiracy  risk  losing  the  job 
(Eden,  Ackermann,  &  Williams,  2005): 

It  is  common  for  commercial  considerations  to  lead  to  “doc¬ 
toring”  of  the  estimate  in  order  to  drive  estimated  costs 
down— particularly  where  there  are  strategic  reasons  for  want¬ 
ing  to  win  that  particular  bid.  Later,  at  the  planning  stage,  this 
“doctoring”  is  forgotten  and  unrealistic  plans  are  made.  As  the 
project  unfolds,  this  lack  of  realism  is  very  likely  to  play  one  of 
the  most  significant  and  unattributed  roles  in  increased  costs. 
Underestimating  at  the  planning  stage  is  one  of  the  most  com¬ 
mon  triggers  for  cost  escalation (p.  19) 

Kahneman  (2011)  argues  that  the  optimism  bias  is  inherent  and 
pervasive  in  individuals  and  teams  taking  risks  under  conditions  of 
uncertainty  or  ambiguity.  He  offers  that  remedies  to  this  bias  are  often 
gained  through  using  a  comparison  of  timelines  for  similar  prior  projects 
as  a  baseline  for  the  current  one,  or  getting  an  outside  view  from  a  third 
party  who  maybe  able  to  assess  the  reasonableness  of  project  estimates. 
He  also  encourages  the  practice  of  “pre-mortems,”  where  the  project 
team  envisions  future  project  failure  and  offers  all  the  things  that  might 
have  caused  it  (p.  264).  This  exercise  may  empower  the  team  to  bring  to 
light  issues  that  have  not  been  previously  considered  (or  ignored),  help 
break  groupthink,  and  encourage  the  team  to  accept  evidence  of  overop¬ 
timism  in  the  project’s  planning. 

Conclusions 

While  the  quantitative  evidence  linking  schedule  to  cost  is  murky, 
most  of  the  literature  agrees  qualitatively  that  longer  programs  incur 
greater  costs  and  cost  overruns.  Longer  programs  tend  to  be  more  com¬ 
plex  and  include  significant  technology  development  efforts.  They  are 
more  susceptible  to  requirements  changes  and  budget  churn.  Longer 
programs  seem  to  have  an  affective  component  where  time  is  under¬ 
valued  and  decisions  can  be  deferred.  Pushing  work  into  the  future  can 
create  a  bow  wave  of  work  that  must  then  be  accomplished  with  more 
concurrency,  thereby  generating  the  need  to  apply  additional  resources 
in  an  attempt  to  meet  the  program  delivery  date.  This  limits  the  pro¬ 
gram’s  cost-schedule-performance  trade-space  and  can  lead  to  even 
greater  churn. 
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Several  potential  remedies  have  been  suggested  by  the  literature. 
First,  where  possible,  shorten  program  timelines  from  inception  to 
delivery  of  a  usable  capability  (a  goal  should  be  no  longer  than  Q-'/2  years). 
This  would  expose  higher  risk  requirements  and  technology  develop¬ 
ment  that  could  be  done  within  a  program  and  reduce  the  opportunities 
for  requirements  changes.  Shortened  programs  would  facilitate  more 
accurate  cost  and  schedule  estimates,  and  fewer  budget  cycles  would 
limit  the  opportunities  for  comptrollers  and  Congress  to  change  program 
funding  profiles. 

Next,  artificial  program  “need  dates”  should  be  compared  with  a 
rational  bottom-up  schedule  derived  from  the  WBS  and  systems  engi¬ 
neering  process.  The  bottom-up  analysis  should  then  inform  a  more 
reasonable  program  delivery  date  and  moderate  the  amount  of  pro¬ 
gram  concurrency.  Similarly,  overoptimism  must  also  be  tempered  by 
objectively  comparing  current  plans  and  schedules  with  similar  past 
programs.  Further,  from  the  survey  of  senior  defense  PMs,  it  appears,  at 
least  in  principle,  that  they  value  and  appreciate  the  utility  of  good  inte¬ 
grated  schedules.  However,  it  appears  equally  likely  that  inconsistencies 
and  contradictions  exist  in  how  PMs  use  schedules  in  actual  practice. 
These  results  imply  that  there  may  be  a  need  for  better  training  and  a 
renewed  focus  on  schedule  development  and  management.  Finally,  PMs 
and  acquisition  leaders  must  understand  and  appreciate  the  linkage  of 
cost  and  schedule,  and  the  value  of  time  to  program  success.  Time,  after 
all,  is  money. 
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Are  the  Performance 
Based  Logistics  Prophets 
Using  Science  or  Alchemy 
to  Create  Life -Cycle 
Affordability? 

Using  Theory  to  Predict 
the  Efficacy  of  Performance 
Based  Logistics 


f  Wesley  S.  Randall 


Numerous  studies  have  provided  evidence  that  perfor¬ 
mance  based  logistics  (PBL)  can  control  cost  and 
improve  performance.  The  success— and  failure— of  PBL 
strategies  suggest  the  need  to  position  the  PBL  research 
domain  into  a  fabric  of  theory.  Just  as  engineering 
theories  predict  the  reliability  of  a  new  armored  vehicle, 
economic  and  business  theories  provide  a  framework 
that  explains  the  efficacy  of  PBL.  This  article  describes 
the  underlying  theoretical  fabric  of  PBL.  Armed  with 
a  framework  grounded  in  theory,  senior  leaders  can 
make  science-based  decisions  to  explain,  predict, 
refine,  and  advocate  for  affordability-enhancing,  life- 
cycle  governance  structures  by  leveraging  the  critical 
success  factors  of  PBL. 
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In  recent  years,  a  great  debate  has  surrounded  the  effectiveness  of 
Performance  Based  Logistics  (PBL).  Articles  have  extolled  the  virtue 
and  condemned  the  opportunism  of  PBL.  In  some  quarters,  consensus  is 
growing  that  PBL  works  if  you  do  it  correctly  (Boyce  &  Banghart,  2012). 
Initiatives  like  the  DoD  Weapon  System  Acquisition  Reform  Product 
Support  Assessment  (Department  of  Defense  [DoD],  2009),  Proof  Point 
Project  (DoD,  2011),  and  DoD  guidance  tell  us  the  PBL  debate  is  techni¬ 
cally  over.  Yet,  in  other  quarters  there  remains  disagreement  as  to  the 
efficacy  of  PBL. 

PBL  is  part  of  a  family  of  strategies,  such  as  performance  based 
contracting  and  pay  for  performance,  whose  essence  is  a  shift  from  pur¬ 
chasing  discrete  products  and  services  to  the  purchase  of  performance 
(Kim,  Cohen,  &  Netessine,  2007;  Randall,  Nowicki,  &  Hawkins,  2011). 
PBL  success  depends  on  interactions  among  numerous  variables.  For 
instance,  short-term  contracts  generate  quick  wins  in  classic  logistics 
(warehousing,  transportation,  and  inventory),  medium-length  contracts 
improve  purchasing  and  item  management,  but  real  reliability- driven 
affordability  requires  a  longer  term  contract  (Hypko,  Tilebein,  &  Gleich, 
2010;  Randall,  Pohlen,  &  Hanna,  2010).  As  is  true  of  any  science,  PBL 
research  has  been  a  journey  of  discovery.  Predicting  PBL  success  can 
be  summed  up  in  words  frequently  echoed  throughout  the  halls  of  the 
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Defense  Acquisition  University:  “It  depends ...”  The  logical  next  step  in 
PBL,  and  the  goal  of  this  article,  is  to  provide  the  business  and  economic 
science  behind  PBL  success. 


The  most  profound  contribution  of  academic 
research  is  theory.  Theory  provides  the  power  to 
explain,  predict,  and  improve  the  future. 


What  is  Science,  Where  Do  We  Use  It? 

The  most  profound  contribution  of  academic  research  is  theory. 
Theory  provides  the  power  to  explain,  predict,  and  improve  the  future. 
Imagine  an  engineer  tasked  to  build  a  light  armored  vehicle  capable  of 
10,000  miles  without  major  overhaul.  Without  theory  as  a  guide,  how  does 
that  engineer  complete  the  task?  The  engineer  will  likely  look  to  what 
seemed  to  work  in  the  past  and  often  overbuild  the  vehicle.  The  engineer 
will  range-test  the  vehicle,  rework  the  design  of  what  breaks,  and  keep 
going  until  the  goal  is  met.  Armed  with  engineering  theory,  our  intrepid 
engineer  is  focused  and  efficient.  Theory  informs  computer  simulation, 
tests,  refinement,  production,  and  employment  of  a  vehicle  that  works  as 
predicted.  To  a  degree,  PBL  today  is  that  engineer  relying  on  experience 
without  theory.  We  have  identified  where  PBL  has  worked— and  hasn’t 
worked— no  small  task.  Now  is  the  time  to  define  a  theory-based  frame¬ 
work  that  guides  PBL  implementation  and  execution. 

One  of  the  first  classes  in  a  business  PhD  involves  a  philosophy  of 
science.  The  goal  of  this  seminar  is  to  provide  the  aspiring  academic 
a  foundation  to  accept  the  idea  of  business  theory.  Typically,  the  class 
starts  with  theory  from  the  hard  sciences  such  as  engineering  and 
physics  simply  to  lay  out  the  idea  of  theory.  The  hard  sciences  are  used 
because  students  generally  have  find  it  easier  to  accept  the  predictive 
validity  of  natural  laws  like  gravity.  There  is  little  resistance  to  the  idea 
that  engineering  theories  will  explain  how  a  new  material  will  impact 
system  performance  (e.g.,  reliability,  maintainability,  and  supportabil- 
ity).  In  the  hard  sciences,  the  idea  of  innovation  improving  performance 
and  reducing  costs,  even  for  fielded  systems,  is  relatively  concrete. 
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In  the  soft  sciences,  such  as  business  and  economics,  the  explanatory 
ability  of  theory  is  not  so  intuitively  grasped.  This  is  because  business 
and  economic  theories  are  less  precise  in  making  point  predictions,  but 
fairly  precise  in  making  statistical  predictions.  While  business  theory 
will  struggle  to  explain  the  success  or  failure  of  a  particular  firm,  business 
theory  is  effective  in  making  statistical  projections  about  groups  of  firms. 
So  it  is  not  unreasonable  for  business  theory  to  suggest  that  on  average 
firms  that  increase  their  inventory  turn  rates  will  outperform  those  who 
do  not  (Arnold,  2002).  The  need  to  explain  and  predict  general  success  in 
the  marketplace  and  overall  health  of  a  national  economy  make  business 
theory  critical.  Likewise,  the  hard  science  business  theory  takes  some  of 
the  alchemy  out  of  strategy.  Without  sound  business  theory,  leaders  may 
find  themselves  moving  from  one  “fad  strategy”  to  the  next,  not  under¬ 
standing  why  things  work  in  one  context  and  not  the  next. 

What  Does  This  Mean  for  PBL? 

The  DoD  budget  realities  highlight  the  need  for  strong,  theoretically 
based,  business  acumen  in  weapon  system  sustainment.  Such  theory 
acumen  is  critical  to  the  fiduciary  responsibility  of  leaders  charged  with 
stewardship  of  defense  budgets,  warfighter  effectiveness,  and  the  success 
of  the  defense  industrial  base.  The  goal  of  this  article  is  to  lay  a  foundation 
for  that  acumen.  Just  as  engineering  theories  explain  the  reliability  of  the 
new  light  armored  vehicle,  economic  and  business  theories  can  be  used  to 
explain  the  efficacy  of  PBL  and  other  postproduction  support  strategies. 

Theory  Foundation  for 
Performance  Based  Logistics 

In  recent  years,  practical  and  academic  PBL  research  in  defense  and 
beyond  has  experienced  a  veritable  explosion  (Boyce  &  Banghart,  2012; 
Guajardo,  Cohen,  Kim,  &  Netessine,  2012;  Kim,  Cohen,  Netessine,  & 
Veeraraghavan,  2010);  Kratz  &  Diaz,  2012;  Mirzahosseinian  &  Piplani, 
2011;  Randall,  Pohlen,  &  Hanna,  2010;  Sols,  Nowicki,  &  Verma,  2007). 
Organizations  looking  to  the  World  Bank  for  financial  support  to  provide 
healthcare  are  expected  to  use  a  performance  based  toolkit  (The  World 
Bank,  2008).  Siemens  (2011)  has  an  integrated  performance  based  strat¬ 
egy  for  rail  services.  More  than  35  countries  are  using  performance  based 
approaches  for  roads  and  highways  (Transportation  Research  Board, 
2009).  The  State  of  Illinois  has  been  recognized  for  its  performance 
based  approach  to  child  welfare  services  (Administration  for  Children  & 
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Families,  2011).  Nearly  70  percent  of  commercial  maintenance,  repair,  and 
overhaul  functions  employ  performance  based  strategies  (Flint,  2007). 
Poignantly,  our  colleagues  in  the  former  Soviet  Union  provide  concise 
insight  into  the  essence  of  a  performance  based  strategy  (Organisation 
for  Economic  Co-Operation  and  Development,  2011): 

Ultimately,  performance  based  contracts,  if  developed  properly, 
can  help  to  lay  the  basis  for  the  long-term  sustainability  of  water 
utilities,  increasing  their  efficiency  and  creating  conditions 
where  investment  capital  can  be  attracted,  (p.  3) 

What  does  research  say? 

•  PBL  manufactures  internal  competition,  eliminates  waste, 
and  improves  quality  (Boyce  &  Banghart,  2012). 

•  PBL  aligns  incentives  to  avoid  suboptimization  (Randall 
et  al.,  2010). 

•  PBL  leverages  long-term  contracts  to  spur  investment  (Sols, 
Nowicki,  &  Verma,  2007). 

•  PBL  optimizes  management  of  assets  that  are  difficult  to 
predict  statistically  (Kim  et  al.,  2010). 

•  PBL  shifts  from  a  return  on  sales  to  return  on  investment 
business  model  (Randall  et  al.,  2011). 

•  PBL  creates  optimal  outcomes  while  dealing  with  uncer¬ 
tainty  and  differing  constraints  (Kim  et  al.,  2007). 

•  PBL  creates  a  governance  structure  based  upon  long-term 
relationships,  stable  cash  flow,  clear  scope,  and  intelligent 
metrics  (Kratz  &  Diaz,  2012). 

The  net-net  of  this  research  activity  is  performance  based  strate¬ 
gies  work. 
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Governance: 

So  That’s  What  PBL  is  Really  All  About 

Research  funded  by  the  Naval  Postgraduate  School  Acquisition 
Research  Program  and  conducted  by  the  University  of  North  Texas 
Complex  Logistics  Systems  Cluster  (Randall  et  al.,  2011)  found: 

PBL  establishes  a  metric-based  governance  structure  where 
suppliers  make  more  profit  when  they  invest  in  logistics  process 
improvements,  or  system  redesign  that  reduces  total  cost  of 
ownership,  (p.  324) 

Governance  is  critical  to  business  and  economic  theory.  Oliver 
E.  Williamson  won  a  Nobel  Prize  describing  firm  governance.  For 
Williamson  and  his  colleague,  William  G.  Ouchi  (1981),  governance  is  a 
way  of  organizing  transactions.  Governance  is  more  than  a  contract;  it  is 
“a  much  broader  concept  than  control.  Essentially,  governance  includes 
elements  of  establishing  and  structuring  exchange  relationships  as  well 
as  aspects  of  monitoring  and  enforcement”  (Heide,  1994,  p.  72).  The 
essence  of  PBL  is  a  governance  mechanism  that  efficiently  organizes 
complex  supply  chain  transactions.  Just  as  the  efficiency  of  transaction 
“bundling”  predicts  the  success  of  the  firm  (Coase,  1937),  PBL  pro¬ 
vides  a  “consistent  sustainment  governance  process  institutionalizing 
a  life-cycle  perspective  on  affordable  and  effective  product  support  from 
acquisition  through  operations  and  support”  (Kratz  &  Diaz,  2012,  p.  40). 

Coase’s  Theory  of  the  Firm: 

Rationale  for  a  Product  Support  Integrator  (PSI) 

Ronald  Coase,  Williamson’s  mentor,  won  a  Nobel  Prize  (The  Ronald 
Coase  Institute,  n.d.),  by  asking  a  very  elemental  question:  Why  do  firms 
exist?  Using  precise  and  brilliantly  simple  terms,  Coase  (1937)  explained 
that  firms  provide  a  governance  structure  that,  for  some  transactions, 
is  more  efficient  than  market  transactions.  The  firm  does  this  by  avoid¬ 
ing  the  market  costs  associated  with  knowing  true  price,  searching  for 
products  and  information,  and  enforcing  contracts.  Value  is  created 
when,  “within  a  firm,  . .  .  market  transactions  are  eliminated,  and  in 
place  of  the  complicated  market  structure  with  exchange  transactions 
is  substituted  the  entrepreneur-coordinator,  who  directs  production” 
(Coase,  1937,  p.  388).  For  Coase,  the  firm  is  an  entrepreneur  efficiently 
bundling  and  integrating  market  transactions. 
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The  ideas  embedded  in  the  Theory  of  the  Firm  provide  a  foundation 
to  explore  the  contention  that  multiyear  PBL  contracts  are  monopo¬ 
lies.  Long-term  firm  existence  demonstrates  that  nonpure  competition 
governance  structures  can  provide  value  superior  to  frequent  market 
competition.  More  specifically,  competitive  position  is  then  based  upon 
the  firm’s  ability  to  integrate  complex  transactions  more  efficiently  than 
what  a  customer  could  achieve  in  the  market  alone.  The  firm  governance 
is  a  form  of  “internal  competition,”  which  uses  profit  as  a  source  of  learn¬ 
ing  (Hunt,  2000).  When  the  PBL  governance  results  in  portions  of  profit 
being  reinvested  into  innovation  that  drives  future  profit,  the  multiyear 
PBL  creates  internal  competition  where  profit  leads  to  learning  that 
increases  affordability  (Randall  et  al.,  2010;  Randall  et  al.,  2011). 


The  critical  nuance  that  is  often  misunderstood  by 
competition  advocates  is  that  profit,  not  competition, 
provides  the  signal  that  allows  firms  to  learn. 


The  critical  nuance  that  is  often  misunderstood  by  competition 
advocates  is  that  profit,  not  competition,  provides  the  signal  that  allows 
firms  to  learn.  Integrating  complex  transactions  then  is  the  key  to  the 
efficacy  of  the  PLB  strategy  profit— learning  cycle.  Randall  and  his  col¬ 
leagues  (2010)  found: 

[that  the]  integrator  acts  as  the  network  entrepreneur,  bun¬ 
dling  knowledge  and  capital  resources  to  achieve  the  end  user’s 
requirements  . . .  integration  links  achievement  of  an  outcome 
with  network  members’  actions,  (p.  43) 

Effective  PBL  strategies  demonstrate  that  monopoly  is  not  synony¬ 
mous  with  opportunism.  Good  PBL  governance  structure  can  mitigate 
potential  opportunism  by  aligning  profit-based  incentives  (Guajardo  et 
al.,  2012). 
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Transaction  Cost  Economics: 
Explaining  Integrated  Supply  Chain 
Management  and  Long-Term  Contracts 

Coase’s  student,  Oliver  Williamson,  was  also  a  Nobel  Laureate. 
Williamson  (1971, 1975)  wondered  why  megafirms  did  not  vertically 
integrate  complete  markets.  In  his  research,  Williamson  identified 
behavioral  dimensions,  which  he  labeled  as  bounded  rationality,  that 
limit  the  quantity  of  transactions  a  firm  could  effectively  bundle.  At  a 
certain  point,  the  coordination  of  transactions  inside  the  firm  becomes 
so  complex,  decision-maker  capacity  is  limited  by  bounded  rationality, 
and  additional  transactions  result  in  disproportionate  cost  (Rindfleisch 
&Heide,  1997;  Williamson,  1975).  Further,  this  bounded  rationality  was 
proportional  to  the  uncertainties  and  complexity  associated  with  trans¬ 
actions  (Rindfleisch  &  Heide,  1997). 

Bounded  rationality  explains  the  success  of  an  integrated  supply 
chain  network  when  the  transactions  are  complex.  Weapon  systems 
sustainment  strategies  are  tremendously  complex.  Transaction  Cost 
Economics  (TCE)  suggests  that  complicated  tasks  require  an  expert 
integrator  and  a  diverse  network  of  supply  chain  partners  who  have  the 
decision-making  capacity  to  avoid  bounded  rationality  for  their  sub¬ 
system  (Kim  et  al.,  2010;  Randall  et  al.,  2011;  Williamson,  2008).  At  the 
same  time,  bounded  rationality  explains  how  some  more  easily  bundled 
transaction  sets,  such  as  an  organic  depot  returning  a  part  to  specifica¬ 
tion,  can  be  more  cost-effective. 

A  second  element  of  TCE  relevant  to  PBL  is  opportunism.  Williamson 
(1985,  p.  47)  defined  opportunism  as  “self-interest  seeking  with  guile.” 
Transactional  relationships  have  little  safeguard  against  opportunis¬ 
tic  behavior.  The  governance  structure  of  the  firm  avoids  opportunism 
associated  with  internal  transactions  by  creating  convergent  goals,  con¬ 
trolling  activities,  and  rewarding  success  (Rindfleisch  &  Heide,  1997). 
If  the  PBL  strategy  is  considered  a  “firm-like  unit”  for  the  purpose  of 
governance,  then  TCE  can  be  used  to  explain  how  metrics  and  long-term 
contracts  create  convergent  goals,  control  activities,  and  reward  goal 
achievement  through  profit.  For  complex  transactions,  the  PBL  supplier 
network,  working  under  sound  governance,  aligns  metrics  and  profit  to 
provide  a  learning  process  superior  to  frequent  competition  and  mini¬ 
mizes  the  effects  of  bounded  rationality. 
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In  transactional  sustainment,  little  incentive 
exists,  and  even  less  capital  is  available,  to  make 
life-cycle  affordability  investments.  PBL  reverses 
that  trend  by  treating  repair  and  redesign  similar 
to  make  or  buy. 


Make  or  Buy  Decisions:  Know  When  to  Hold  ’em 
(Repair)  and  Know  When  to  Fold  ’em  (Redesign) 

A  defining  element  of  a  complex  system  is  one  where  the  postproduc¬ 
tion  spend  significantly  exceeds  the  production  spend.  Unfortunately, 
that  postproduction  spending  at  best  simply  maintains  the  status  quo.  In 
transactional  sustainment,  little  incentive  exists,  and  even  less  capital 
is  available,  to  make  life- cycle  affordability  investments.  PBL  reverses 
that  trend  by  treating  repair  and  redesign  similar  to  make  or  buy. 

Simply  put,  make  or  buy  predicts,  when  all  transactions  costs  are 
considered,  if  firms  should  make  or  buy  an  item  (Coase,  1937;  Walker 
&  Weber,  1984;  Williamson,  1985,  2008).  The  make-or-buy  decision 
seeks  “the  most  efficient  mode  of  governing  the  transaction”  (Walker 
&  Weber,  1984,  p.  373).  The  idea  underlying  the  choice  of  make  or  buy  is 
similar  to  the  spare-or-repair  decisions  predicted  by  the  PBL  governance 
structure.  The  goal  in  make  (repair)  or  buy  (redesign)  is  to  seek  the  most 
cost-efficient  approach  to  satisfy  demand  for  some  item. 

PBL  and  non-PBL  managers  focus  on  gaining  efficiency  and  effective¬ 
ness  regarding  inventory  management,  repair,  and  overhaul.  Yet,  for  the 
PBL  manager  the  money  spent  purchasing  spares,  repairs,  and  overhaul 
is  continuously  calculated  against  an  investment  in  new  materials, 
processes,  and  technologies  that  will  improve  reliability  and  correspond¬ 
ingly  drive  out  demand  for  that  particular  spare  part  (and  its  warehouse, 
inventory,  and  transportation  cost),  along  with  repair  or  overhaul  tasks 
(Randall  et  al.,  2011). 

When  a  supplier  has  a  new  process  that  reduces  the  cost  to  redesign 
(buy)  a  part,  then  the  PBL  strategy  dictates  a  shift  from  repair  to  rede¬ 
sign.  Figure  1  graphically  depicts  using  the  idea  of  make  or  buy  for  repair 
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or  redesign.  The  vertical  axis  denotes  the  financial  case  for  repair  or 
redesign.  The  horizontal  axis  represents  different  parts.  The  parts  that 
appear  on  the  left  and  right  side  have  a  clear  financial  case.  The  ability  of 
the  PBL  governance  to  use  innovation  and  investment  to  move  the  parts 
near  the  middle  from  repair  to  redesign  is  the  essence  of  life-cycle  afford¬ 
ability.  In  transactional  postproduction  support,  there  is  no  governance 
mechanism  to  shift  the  repair-redesign  frontier. 

The  long-term  contracts  create  pools  of  monetized  cost  avoidance 
that  represent  potential  profit  when  the  repair-redesign  frontier  is 
moved  to  the  right.  The  profit  pools  provide  the  suppliers  an  incentive 
to  invest  in  new  material,  process,  and  capabilities  that  will  push  items 
across  the  repair-redesign  efficient  frontier.  Thus  the  learning— invest¬ 
ment— profit  cycle  of  PBL  overcomes  the  potential  opportunism  of 
limited  external  competition. 

FIGURE.  GRAPHIC  DEPICTION  OF  THE  IDEA  OF  MAKE  OR  BUY 

FOR  REPAIR  OR  REDESIGN 


STRONG  STRONG 


The  idea  that  new  materials,  processes,  and  technologies  will  move 
parts  across  the  redesign  frontier  is  fairly  intuitive.  The  economic  case 
for  shifting  from  repair  to  redesign  is  less  intuitive.  The  costs  associated 
with  redesign  can  be  daunting;  they  include  the  engineering  hours  of  the 
design  itself,  production  of  parts,  changes  to  the  configuration  baseline 
and  technical  orders,  new  test  equipment,  and  new  spares.  However,  the 
cost  associated  with  continuous  repair  is  very  real.  Ultimately,  life-cycle 
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cost  reduction  requires  managers  to  recognize  a  compelling  case  for  the 
recovery  of  nonrecurring  costs  linked  to  redesign.  The  math  associated 
with  redesign  is  fairly  surprising. 

Table  1  provides  a  redesign  economic  model.  This  example  illumi¬ 
nates  the  impact  that  contract  length  has  on  the  repair-redesign  decision. 
Targeting  demand  reduction  on  a  few  key  parts  can  have  significant 
impact  on  affordability.  This  example  assumes  constant  year  dollars, 
and  no  weighted  average  cost  of  capital  discount.  Given  that  a  $7  mil¬ 
lion  component  redesign  doubles  the  mean  time  between  failure,  and 
that  doubling  reduces  demand  for  that  component  by  50  percent  (200  to 
100  demands  per  year),  a  rational  actor,  given  a  3-year  contract,  will  not 
invest  the  $7  million  in  nonrecurring  redesign  costs.  However,  extending 
the  contract  by  1  year,  we  find  that  a  rational  actor  will  make  the  rede¬ 
sign  investment.  Universally,  when  a  component  switches  from  repair 
to  redesign,  both  the  near-term  (on-equipment  maintenance)  costs  and 
long-term  (total  life-cycle)  costs  go  down  for  the  customer.  Once  the 
nonrecurring  costs  are  recouped,  the  cost  avoidance  piles  up  for  the 
remaining  life  of  the  system— increasing  affordability. 


TABLE  1.  IMPLICATION  OF  CONTRACT  LENGTH  ON  REPAIR 
VERSUS  REDESIGN  EFFICIENT  FRONTIER 


Repair 

Redesign 

Demand 

200 

100 

Cost  per  repair 

$20,000 

$20,000 

Nonrecurring 

$0 

$7,000,000 

Total  cost  year  1 

$4,000,000 

$2,000,000 

Cumulative  cost 

$4,000,000 

$9,000,000 

Total  cost  year  2 

$4,000,000 

$2,000,000 

Cumulative  cost 

$8,000,000 

$11,000,000 

Total  cost  year  3 

$4,000,000 

$2,000,000 

Cumulative  cost 

$12,000,000 

$13,000,000 

Total  cost  year  4 

$4,000,000 

$2,000,000 

Cumulative  cost 

$16,000,000 

$15,000,000 

Total  cost  year  5 

$4,000,000 

$2,000,000 

Cumulative  cost 

$20,000,000 

$17,000,000 

Cost  Savings  Redesign 

$3,000,000 

Note.  For  simplicity,  analysis  does  not  consider  discounted  net  present  value. 
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In  PBL,  contract  length  directly  impacts  the  repair- redesign  efficient 
frontier  and  has  tremendous  implications  for  life-cycle  affordability.  The 
length  of  a  specific  PBL  contract  depends  on  the  potential  to  drive  out 
cost  by  redesign,  recovering  nonrecurring  costs,  and  the  costs  associated 
with  repair.  This  raises  the  question,  is  there  an  end  point  where  no  more 
cost  can  be  avoided?  In  theory,  yes;  in  practice,  no.  The  key  determinants 
for  success  will  be  the  availability  of  new  processes,  materials,  and 
technologies,  and  the  ability  to  monetize  out -year  spending.  As  long  as 
innovation  and  supply  chain  collaboration  results  in  a  shift  from  repair 
to  redesign,  and  as  long  as  defense  postproduction  spending  accounts 
for  billions  of  dollars,  PBL-type  strategies  will  continue  to  produce  effi¬ 
ciency  and  effectiveness  improvements. 

Core  Competency: 

Exactly  Who  Should  Do  What  and  Why? 

Significant  PBL  discussion  is  still  ongoing  about  who  should  do 
what  and  why.  A 1990  Harvard  Business  Review  article  by  Prahalad  and 
Hamel  (1990)  provides  a  theory-based  way  to  answer  that  question.  They 
discuss  the  idea  of  core  competency  as  a  framework  to  integrate  orga¬ 
nizations  that  have  the  complementary  “core  competencies”  needed  to 
achieve  success.  Core  competencies  are  “the  most  powerful  way  to  pre¬ 
vail  in  global  competition”  (Prahalad  &  Hamel,  p.  79).  Core  competencies 
are  central  to  business  success.  By  definition,  the  core  competencies  of  a 
firm  are  difficult  to  imitate,  versatile  in  the  marketplace (s),  and  protect 
against  commoditization  by  being  recognizable  as  significant  value  to 
the  end  customer  (Prahalad  &  Hamel,  1990).  Recognizing  what  core 
competencies  are  and  are  not  allows  business  strategists  to  figure  out 
when  to  partner,  and  who  should  do  what. 

The  idea  of  core  competency  should  drive  the  teaming  strategy  of 
the  PBL  governance  structure.  For  example,  generally,  the  entity  that 
designed  and  produced  the  part  will  have  the  highest  redesign  core  com¬ 
petency.  When  a  third  party  offers  a  lower  cost  solution,  this  is  likely  to 
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be  the  result  of  some  externality  (e.g.,  the  third  party  wanting  into  the 
market,  the  third  party  developing  some  type  of  Schumpeterian  innova¬ 
tion,  the  original  equipment  manufacturer  [OEM]  losing  competency,  or 
an  inefficient  OEM  cost  structure).  The  competency  logic  also  applies  to 
purchasing,  inventory  management,  warehousing,  and  transportation. 
Core  competency  explains  why  highly  successful  third-party  logistics 
providers,  like  Menlo  Logistics,  are  valued  DoD  partners.  When  it  comes 
to  repair  and  overhaul,  the  DoD  depots  have  established  core  competen¬ 
cies  that  often  make  them  the  smart  partner  of  choice.  Few  would  argue 
the  ability  of  Tinker  and  Jacksonville  to  overhaul  engines,  or  Ogden  to 
rebuild  landing  gear.  Core  competency  provides  a  theory-based  frame¬ 
work  for  decision  makers  to  predict  who  should  be  doing  what  and  why. 
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It  makes  sense  that  suppliers  would  hear  the  risk 
for  performing  typical  logistics  functions,  while 
also  making  the  repair-redesign  decision.  Yet,  risk 
management  is  about  balancing  risk  and  reward. 


It’s  About  Risk 

Risk  allocation  and,  more  specifically,  placing  risk  where  it  is  han¬ 
dled  most  cost-effectively  is  elemental  to  PBL  (Randall  et  al.,  2011): 

PBL  represents  a  governance  structure  that  drives  responsibility  for 
supply  chain  transactions  to  those  entities  most  capable  of  completing 
those  transactions  at  the  least  cost  and  lowest  risk.  (p.  343) 

It  makes  sense  that  suppliers  would  bear  the  risk  for  performing  typi¬ 
cal  logistics  functions,  while  also  making  the  repair-redesign  decision. 
Yet,  risk  management  is  about  balancing  risk  and  reward.  Not  uncom¬ 
monly,  PBL  contracts  may  specify  some  type  of  gain  sharing  when  profits 
exceed  a  certain  level,  subject  to  investments  having  been  recovered.  At 
the  same  time,  it  may  be  appropriate  that  customers  share  in  costs  asso¬ 
ciated  with  unforeseeable  circumstances.  Both  ends  of  the  risk-reward 
spectrum  can  be  addressed  by  the  governance  structure. 

Supply  Chain  Management:  On  the  Rise 

In  a  landmark  2001  Supply  Chain  Management  Review  article,  Rice 
and  Hoppe  (2001)  argued  that  competition  is  no  longer  firm  against  firm, 
but  supply  chain  versus  supply  chain.  Improved  connectivity,  increased 
efficiency,  higher  quality,  and  standardized  processes  have  reduced  sup¬ 
ply  chain  transaction  costs  (Kaipia,  2009).  Supply  chain  management 
efficiently  brings  together  firms  with  complementary  core  competencies. 

How  significant  is  the  rise  in  supply  chain  management? 
Manufacturers  spend  40  to  70  percent  of  the  cost  of  goods  on  purchased 
goods  and  services  (Trent,  2007).  The  efficiency  of  supply  chain  gov¬ 
ernance  structures  and  the  ability  to  integrate  complementary  core 
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competency  have  resulted  in  transactions  moving  from  the  firm  to  the 
supply  chain.  Even  Walmart  goes  to  the  supply  chain  for  logistics  support— 
Exel  Logistics  manages  Walmart’s  Canadian  logistics  operations.  Bose  has 
heavily  integrated  suppliers  into  its  research,  design,  and  manufacturing. 

Effective  supply  chain  management  can  deliver  products  and  ser¬ 
vices  that  create  customer  value  at  the  least  total  cost.  Supply  chain 
management  value  (outcome  divided  by  cost)  is  based  upon  the  ability 
of  the  integrated  supply  chain  to  exceed  the  value  of  internally  managed 
transactions  (Lambert  &  Garcia-Dastugue,  2006;  Walker  &  Weber,  1984, 
1987).  The  rise  of  efficient  supply  chain  governance  structures  is  not 
coincidental  to  the  rise  of  PBL  success. 


Providing  value  is  inherently  a  return  on 
investment  strategy  and  requires  a  long-term 
relationship  between  customers  and  suppliers. 


Service-Dominant  Logic:  A  New  Exchange 
Framework— And  It  Looks  a  Lot  Like  PBL 

Scholars  Stephen  Vargo  and  Bob  Lusch  (2004)  recently  developed 
a  new  framework  for  market  exchange  called  Service-Dominant  Logic 
(SDL).  SDL  suggests  that  economic  expansion  and  competitive  position 
can  be  predicted  based  upon  the  supplier  networks’  ability  to  leverage 
knowledge  to  create  evolving  customer  value.  Similar  to  PBL,  SDL 
creates  value  not  by  delivering  products,  but  by  using  knowledge  to 
create  performance. 

In  SDL,  the  product  is  not  in  and  of  itself  valuable— the  product  is 
a  distribution  mechanism  for  value  (Lusch,  2011).  This  value  focus  is 
similar  to  PBL.  In  PBL,  the  metric  provides  a  value-based  feedback 
mechanism.  Specifying  delivery  of  performance,  not  products,  leaves  the 
suppliers  free  to  invest  in  innovation,  create  cost  avoidance,  and  harvest 
profits.  This  dynamic  creates  learning,  rewards  investment,  and  spurs 
new  investment.  Similarly,  in  SDL  the  primary  flow  is  knowledge,  and 
integration  is  the  highest  core  competency. 
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The  idea  of  PBL  as  an  application  of  SDL  has  received  consider¬ 
able  attention  in  academic  circles.  A  2011  PBL-focused  article  in  the 
International  Journal  of  Logistics  Management  won  the  emerald  literati 
commendable  paper  award  (Randall  et  al.,  2011).  That  article  states 
that  SDL: 

.  . .  provides  an  effective  mechanism  to  show  how  certain  PBL 
ecosystems,  their  suppliers,  customers,  and  integrator,  can 
efficiently  adapt  to  environmental  changes,  and  thus  predict 
competitive  advantage  of  that  network.  The  key  to  that  com¬ 
petitive  advantage  is  the  flow  of  knowledge-based  resources 
between  the  supplier  network  partners  as  focused  on  satisfying 
a  customer  service  requirement,  (p.  332) 

Providing  value  is  inherently  a  return  on  investment  strategy  and 
requires  a  long-term  relationship  between  customers  and  suppliers. 
The  knowledge  orientation  of  SDL  and  its  focus  on  creation  of  value,  not 
simply  supplying  product,  provides  an  economic  foundation  to  predict 
the  success  of  a  PBL  governance  structure  that  aligns  metrics,  incentive, 
knowledge  management,  integration,  capital,  supply  chain  relationships, 
and  learning  to  create  affordability. 


Research  should  seek  to  develop  design  solutions, 
coupled  with  efficacious  PBL  governance  structure, 
thereby  enabling  cost-effective  innovation  across  a 
program’s  life. 


A  Framework  for  PBL  Governance  Decisions 

PBL  rests  on  a  fabric  of  sound  business  and  economic  theory.  PBL 
governance  structures  minimize  the  costs  associated  with  filling 
demand  for  parts,  while  continuously  reevaluating  how  new  material, 
processes,  and  technologies  can  improve  reliability  and  repair  efficiency, 
reduce  demand  for  parts,  and  decrease  life-cycle  cost. 
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This  implies  a  powerful,  yet  fairly  simple  PBL-based  Life  Cycle 
Affordability  framework  (Table  2).  Given  a  proper  PBL  governance 
structure,  affordability  can  be  achieved  by  reducing  the  supply  chain 
cost  associated  with  meeting  demand  for  parts  (X-axis)  or  reducing  the 
demand  for  parts  and  cost  of  repair  (Y-axis).  Program  characteristics 
(e.g.,  parts  demand)  can  be  used  to  determine  differing  potential,  con¬ 
tract  structure,  and  partnerships— governance. 


TABLE  2.  LIFE-CYCLE  AFFORDABILITY  FRAMEWORK 


Cost  avoidance  potential  -  Medium 

•  Demand  for  parts  is  low 

•  Demand  for  repairs  is  low 

•  Redesign  potential  is  high 

Potential  opportunities: 

•  Reliability,  repair,  and  diagnostic 

Cost  avoidance  potential  -  High 

•  Demand  for  parts  is  high 

•  Demand  for  repairs  is  high 

•  Redesign  potential  is  high 

Potential  opportunities: 

•  Supply  Chain,  reliability,  repair,  and 
diagnostic 

Cost  avoidance  potential  -  Low 

•  Demand  for  parts  is  low 

•  Demand  for  repairs  is  low 

•  Low,  or  risky  redesign  potential 

Potential  opportunities: 

•  Limited 

Cost  avoidance  potential  -  Medium 

•  Demand  for  parts  is  high 

•  Demand  for  repairs  is  high 

•  Low,  or  risky  redesign  potential 

Potential  opportunities: 

•  Supply  chain 

LOW 


Supply  chain  integration  potential 


HIGH 


Quite  simply,  how  quickly  and  cost-effectively  new  materials,  pro¬ 
cesses,  and  technologies  are  infused  into  a  PBL  program  is  the  essence  of 
affordability.  Research  should  seek  to  develop  design  solutions,  coupled 
with  efficacious  PBL  governance  structure,  thereby  enabling  cost-effec¬ 
tive  innovation  across  a  program’s  life. 
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Summary 

This  article  uses  business  and  economic  theory  to  weave  a  theo¬ 
retical  framework  that  gives  leaders  the  ability  to  explain,  predict, 
refine,  and  advocate  for  effective  PBL  strategy.  The  theory  of  the  firm 
provides  a  mechanism  to  the  role  of  an  integrator  to  act  as  the  network 
entrepreneur  who  reduces  transaction  cost  and  efficiently  links  actions 
with  outcomes.  TCE  affirms  the  role  of  integration,  while  describing  in 
theoretical  terms  how  PBL  governance  addresses  bounded  rationality 
and  opportunism.  Bounded  rationality  and  the  idea  of  core  competency 
explain  why  PBL  strategies  benefit  from  the  network  of  firms  collaborat¬ 
ing  to  increase  affordability.  Understanding  opportunism  gives  insight 
into  how  monetizing  cost  avoidance  ameliorates  the  negative  aspects  of 
a  monopoly  partnership.  Ultimately,  profit  leads  to  learning,  and  learn¬ 
ing  leads  to  smart  investment— thus  profit,  learning,  and  investment 
cannot  be.  DoD  can  continue  to  spend  on  spares,  repairs,  and  overhaul, 
or  it  can  create  partnerships  that  leverage  new  materials,  processes, 
and  technologies  and  supplier  investment  to  improve  affordability. 
TCE  shows  how  the  PBL  governance  structure  manufactures  internal 
competition  that  is  more  efficient  than  frequent  market  competition  for 
complex  transactions.  Make  or  buy  explains  how  shifting  from  repair  to 
redesign  is  the  essence  of  affordability,  and  the  role  of  contract  length 
in  that  decision  process. 

Supply  chain  management  is  shown  to  provide  the  complementary 
core  competencies  needed  to  create  affordable  complex  systems.  PBL 
also  uses  the  idea  of  competency  to  drive  risk  to  the  point  where  it  is 
managed  most  cost-effectively.  PBL  is  shown  to  be  a  practical  imple¬ 
mentation  of  the  SDL  exchange  paradigm.  This  means  the  massive 
expansion  of  high-quality,  peer-reviewed  research  into  SDL  research 
provides  a  readymade  foundation  to  further  the  efficacy  of  PBL.  In  PBL 
and  SDL,  what  matters  most  to  customers  is  performance  (service),  not 
parts  (products). 


Conclusions 

The  Proof  Point  Project  (Boyce  &  Banghart,  2012)  provided  empirical 
evidence  of  PBL  success.  This  article  augments  that  effort  by  providing 
the  business  and  economic  theory  at  the  core  of  that  success.  The  critical¬ 
ity  of  reducing  a  weapon  system’s  life  cycle  demands  that  senior  leaders 
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inculcate  into  the  acquisition  corps  a  respect  for  business  theory— simi¬ 
lar  to  the  strong  respect  that  the  corps  has  for  engineering  theory.  It  is 
critical  that  decision  makers  continue  the  intellectual  engagement  aimed 
at  understanding  the  theoretical  and  practical  foundation  for  successful 
PBL  governance  structures. 

Can  we  do  better?  Certainly.  But  let’s  not  forget  what  we  have  done. 
DoD  has  provided  the  most  capable  and  reliable  warfighting  systems  ever 
known.  I  have  personal  experience  with  a  number  of  these  systems,  and  I 
am  awestruck  by  their  capabilities  and  the  competencies  of  the  men  and 
women  who  created  them.  At  the  essence  of  PBL,  we  encounter  familiar 
concepts.  We  know  how  to  team.  We  know  how  to  invest.  We  know  how 
to  blend  core  competencies— sea,  land,  air,  and  space.  We  know  how  to 
innovate.  PBL  simply  provides  a  rational  governance  structure  that 
blends  new  ideas  with  old  ideas  to  create  more  affordable  systems.  The 
Life  Cycle  Affordability  Framework  for  PBL  is  encapsulated  in  Table  2. 
What  is  left  is  a  few  guiding  thoughts  on  how  managers  might  implement 
these  insights  via  the  framework.  Table  2  provides  those  thoughts. 

In  capitalism,  the  metric  for  success  is  profit  =  revenue  -  expense. 
In  DoD,  the  metric  for  success  will  be  quantified  as  capability  ( assets  x 
readiness )  where  capability  =  budget  -  cost.  An  effective  entrepreneurial 
program  leader  will  increase  capability  (i.e.,  lethality,  maintainability, 
and/or  reliability)  by  leveraging  innovation  and  governance  to  lower 
cost.  To  that  end,  grooming  leaders  and  program  integrators  who  func¬ 
tion  as  business-savvy  entrepreneurs  is  essential  to  the  success  of  the 
nation’s  warfighters. 

Business  theory  allows  one  last  prediction.  Leaders  have  a  choice. 
Those  leaders  who  choose  not  to  develop  a  theoretical  understanding 
of  life-cycle  affordability  may  unwittingly  begin  to  resemble  mercurial 
alchemists,  with  a  frustratingly  inconsistent  ability  to  reduce  weapon 
system  life-cycle  cost  or  explain  the  efficacy  of  affordability- oriented 
strategies  like  PBL.  Leaders  who  do  not  understand  theory  will  be  forced 
to  watch  as  their  peers  explain,  predict,  refine,  and  advocate  for  PBL 
success  after  success.  Leaders  armed  with  theory  will  understand  how 
to  employ  PBL  strategy  to  build  collaborative  supply  chain  governance 
structures  that  increase  the  affordability  of  national  security. 
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Contracting  in  expeditionary  operations  is  not  new. 
What  is  new  is  the  scope  and  magnitude  of  the  roles 
that  contracting  and  contractors  play  in  today’s  mili¬ 
tary  operations.  Lack  of  planning  and  sound  contract 
integration  at  the  strategic  level  leads  to  inefficiencies, 
ineffectiveness,  and,  in  many  cases,  outright  fraud.  Annex 
W,  Operational  Contract  Support  Plan,  is  the  overall 
operations  plan  for  Geographic  Combatant  Commands 
and  the  Services  within  the  Adaptive  Planning  and 
Execution  System  framework.  The  authors  propose  an 
Integrated  Planner  and  Executor  (IPE)  model  for  opera¬ 
tional  contract  support  and  its  integration  into  Annex  W 
and  existing  war  planning  systems  by  congressionally 
mandating,  authorizing,  and  funding  IPE  positions  within 
Service  structures.  The  IPE  would  be  vested  with  the 
authority  to  establish,  monitor,  and  manage  Annex  W. 
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Military  organizations  throughout  the  world  are  increasingly  called 
to  perform  missions  and  create  outcomes  that  are  reliant  on  contractor 
support.  In  fact,  contractors  perform  myriad  functions  in  modern,  often 
complex,  military  operations.  Additionally,  the  military  services  are 
subject  to  ever  increasing  scrutiny  and  accountability  to  become  better 
stewards  of  scarce  resources,  to  eliminate  potential  waste,  and  to  reduce 
abuse  of  taxpayer  money  due  to  poor  management,  operational  redun¬ 
dancy  and  duplication  of  effort,  and  outright  corruption. 

Military  Stands  to  Gain  from 
Newest  Initiatives  in  Doctrine 

Because  of  an  increased  reliance  on  contractors  and  recent  demands 
for  improved  accountability  and  performance,  the  authors  contend 
that  international  military  organizations  will  benefit  by  incorporating 
Phase  Zero  Contracting  Operations  (PZCO),  and  strategic  and  integra¬ 
tive  planning  for  contingency  and  expeditionary  operations.  The  PZCO 
concept  has  gained  high-level  attention  as  it  is  now  embedded  in  Joint 
Publication  (JP)  5-0,  Joint  Operation  Planning  (Chairman  of  the  Joint 
Chiefs  of  Staff  [CJCS],  2011),  and  in  JP  4-10,  Operational  Contract 
Support  (OCS)  (CJCS,  2008),  and  currently  under  revision.  Additionally, 
PZCO  protocols  were  proposed  and  published  in  2010,  and  the  concept 
has  gained  popularity  among  military  leaders  seeking  to  improve  mili¬ 
tary  capability  while  following  sound  business  practices  (Yoder,  2010). 
PZCO,  in  essence  and  conceptually,  is  somewhat  already  embedded  in 
recent  doctrine,  specifically  within  JP  5-0  and  JP  4-10.  However,  it  is 
not  being  fully  implemented.  This  article  addresses  shortfalls  in  imple¬ 
mentation,  conceptually  and  pragmatically.  Additionally,  the  authors 
utilize  two  analytical  frameworks— the  Three-Tier  Model  (TTM)  and 
three  pillars  for  integrative  success— to  identify  shortfalls  and  recom¬ 
mendations  for  improvements. 

The  PZCO  concept  for  strategic  leaders  and  planners  is  presented, 
including  the  scope  and  magnitude  of  current  and  future  contractor 
support,  the  need  for  integration  and  coordination  amongst  stakehold¬ 
ers,  key  PZCO  model  constructs,  and  alignment  with  key  aspects  of  the 
Adaptive  Planning  and  Execution  System  (APEX),  which  must  include 
contracting.  Finally,  conclusions  and  recommendations  are  provided  for 
forward-thinking  leaders  and  planners. 
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The  Scope  and  Magnitude  of  Contractor 
Support  in  Expeditionary  Operations 

Contracting  in  expeditionary  operations  is  not  a  new  phenomenon. 
What  is  new  is  the  scope  and  magnitude  that  contracting  and  contrac¬ 
tors  play  in  today’s  military  operations.  For  example,  in  March  2011  the 
Congressional  Research  Service  reported  that  in  the  Central  Command 
Area  of  Responsibility,  the  ratio  of  contractors  to  uniformed  personnel 
supporting  operations  was  .81:1  (Schwartz  &  Swain,  2011).  Even  if  global 
operating  tempos  decline,  many  experts  believe  that  reliance  on  contrac¬ 
tor  personnel  will  remain  at  current  levels,  or  even  grow,  in  relation  to 
the  number  of  uniformed  personnel.  The  New  York  Times  reported  in 
February  2012  that  113,491  contractor  personnel  were  in  Afghanistan 
compared  to  90,000  U.S.  soldiers  (Nordland,  2012).  It  should  be  noted 
that  not  all  contractors  in  theater  were  directly  supporting  Department 
of  Defense  (DoD)  operations  in  that,  for  example,  the  United  States 
Agency  for  International  Development  and  many  private  volunteer 
organizations  and  nongovernmental  organizations  utilize  contractors 
and  may  be  included  in  The  New  York  Times- reported  tally. 

Particularly  noteworthy  is  the  scope  and  variety  of  contracted  func¬ 
tions.  These  functions  include  base  operations  support,  weapon  systems 
support,  security  services,  and  a  host  of  others. 


Based  on  continued  public  and  political  pressure 
to  keep  organic  uniform  force  structures  low,  the 
continued  reliance  on  contract  support  for  military 
operations  is  not  likely  to  wane. 


High  Reliance  on  Contracted  Support 
Has  Created  Challenges 

Based  on  continued  public  and  political  pressure  to  keep  organic 
uniform  force  structures  low,  the  continued  reliance  on  contract  sup¬ 
port  for  military  operations  is  not  likely  to  wane.  Nevertheless,  this  high 
reliance  on  contractor  support  has  also  created  challenges  for  military 
planners,  operators,  contracting  units,  and  even  for  the  contractors 
themselves.  Challenges  have  manifested  in  command  and  control,  in 
integration  into  Geographic  Combatant  Command  (GCC)  battle  and 
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operational  schema,  and  in  the  need  for  advanced  planning,  phasing, 
and  timing  of  contracting  events  to  synchronize  with  and  complement 
operations  plans  (OPLAN).  Additionally,  planners  must  consider  com¬ 
munications  and  movement  plans,  weapons  control,  compliance  with 
Host  Nation  and  Status  of  Forces  Agreements  (SOFA),  contract  manage¬ 
ment  and  oversight,  indemnity  and  insurance  of  government-contracted 
personnel,  prevention  of  human  trafficking,  third-country  national  labor 
protections,  issuing  and  maintaining  security  clearances,  and  lawsuits 
under  the  Defense  Base  Act,  to  name  only  a  few.  Many  of  the  challenges 
stem  from  a  shift  in  organic  uniformed-force  capability  to  a  contracted 
capability— from  “doing”  to  “managing.”  So  what  can  military  leaders 
and  planners  do  to  effectively  and  efficiently  manage  all  of  these  aspects 
of  contracted  support?  The  incorporation  of  PZCO  into  the  design  and 
construct  of  military  planning  will  address  many  of  the  challenges 
identified  previously. 
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Credentialed  Contract  Planners  Integrated  with 
Operations  Planners  and  Stakeholders 

The  TTM,  a  credential-based  personnel  hierarchy  for  contracting 
officers  and  planning  staff,  was  published  to  address  the  challenges 
inherent  in  contracting  in  complex  military  operations  (Yoder,  2004). 
It  optimizes  the  integrative  planning,  coordination,  and  execution 
required  for  contingency  and  expeditionary  operations  at  the  tacti¬ 
cal,  operational,  and  strategic  levels  of  the  organization.  The  model  is 
based  on  two  primary  premises.  First,  mission  optimization  occurs  only 
with  well-credentialed  contracting  planners  and  executors.  Second, 
optimized  stakeholder  integration,  including,  for  example,  operational 
commanders,  supporting  units,  nongovernmental  organizations  (NGO), 
and  private  voluntary  organizations  (PVO),  can  only  be  accomplished  by 
utilizing  well-credentialed  participants  in  the  planning  and  execution 
phases  (Yoder,  2011). 

Phase  Zero  Contracting  Operations— 'The  Three-Tier  Model 

The  TTM  has  specific  personnel  credentials  in  three  primary  tiers: 
(a)  Tier  One— Training  and  education;  (b)  Tier  Two— Certification 
(such  as  Defense  Acquisition  Workforce  Improvement  Act  [DAWIA] 
contracting  levels,  security  clearance  requirements,  etc.);  and  (c)  Tier 
Three— Experience. 

The  three  tiers  are  described  in  the  following  paragraphs. 

In  Tier  One,  the  ordering  officer  serves  at  the  lowest  level.  This  con¬ 
tracting  level  has  several  identifying  attributes.  Tier  One  personnel 
reside  within  the  tactical  level  of  the  military  hierarchy  and  are  the  most 
prevalent  contracting  personnel  within  most  formal  military  and  civilian 
organizations.  The  Tier  One  personnel  are  junior  civilians  and  military 
staff.  They  operate  at  the  tactical  and  unit  levels,  and  perform  no  integra¬ 
tive  planning  at  the  operational  and  strategic  levels.  Tier  One  personnel 
place  basic  orders  and  conduct  simple  transactions.  In  the  broadest  terms, 
little  stakeholder  integration  is  being  initiated  or  managed  at  this  level. 
However,  this  lowest  level  is  absolutely  essential  because  it  represents 
where  a  majority  of  “in-the -field”  contracting  actions  are  conducted.  Tier 
One  is  the  tactical  level  of  the  enterprise.  Particular  importance  at  Tier 
One  is  placed  on  standardized  training— emphasizing  protocols,  ethical 
conduct,  management,  control,  and  oversight. 
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In  the  middle  of  the  hierarchy  is  Tier  Two,  which  leverages  the 
capabilities  of  contracting  officers  who  serve  at  the  operational  level. 
The  Tier  Two  personnel  require  enhanced  credentials.  These  personnel 
conduct  complex  contracting  transactions  and  leverage  local  economy 
assets.  They  may  perform  all  functions  associated  with  Tier  One  person¬ 
nel,  but  with  increased  credentials,  scope,  and  responsibilities.  The  TTM 
calls  for  Tier  Two  personnel  to  be  mid -level  civilians,  mid-grade  officers, 
or  credentialed  senior  enlisted.  They  can  be  integrated  into  planning  and 
local  operations,  performing  some  integrative  planning  at  the  tactical 
and  operational  levels;  and  they  can  perform  some  liaison  functions 
with  broader  stakeholders.  Their  main  mission  is  to  optimize  local 
operations  in  harmony  with  strategic  guidance.  They  will  also  prepare 
Annex  W,  Operational  Contract  Support  Plan,  when  serving  on  a  Joint 
Task  Force  staff  during  crisis  action  planning.  Since  Tier  Two  person¬ 
nel  serve  at  the  operational  level  of  the  organization,  expertise  in  the 
protocols,  ethical  conduct,  management,  control  and  oversight,  conduct 
of  complex  negotiations,  broad  business  acumen  in  complex  military 
contracting,  and  phase  I  Joint  Professional  Military  Education  (JPME I) 
is  required.  Currently,  the  Naval  Postgraduate  School  and  the  Defense 
Acquisition  University  offer  CON  234-Contingency  Contracting  and 
CON  334-Advanced  Contingency  Contracting  courses  to  standardize 
education  in  the  contingency  contracting  business  field. 


An  IPE  must  be  strategically  positioned  within 
the  organization  to  achieve  the  highest  levels  of 
integrative  planning. 


The  highest  and  most  crucial  tier  in  the  TTM  is  Tier  Three,  the  flag 
officer  or  senior  civilian  position  designated  as  the  integrated  planner 
and  executor  (IPE).  The  Tier  Three  personnel  are  at  the  strategic  level 
of  military  and  civilian  organizations.  Tier  Three  calls  for  the  high¬ 
est  credentials  including,  but  not  be  limited  to,  JPME  I  &  II,  DAWIA 
Contracting  Level  III  certification  and  warrant  (or  international  equiva¬ 
lent),  a  graduate  degree  or  higher,  a  Top  Secret  security  clearance,  and 
experience  in  operations  and  contracting  gained  through  experiential 
tours  or  assignments  (Yoder,  2010).  Figure  1  highlights  the  key  aspects 
of  the  IPE  position  (Yoder,  2011). 


355 


Defense  ARJ,  October  2013,  Vol.  20 No.  3:349-372 


Phase  Zero  Contracting  Operations  (PZCO) 


FIGURE  1.  THREE-TIER  MODEL:  TIER  THREE,  INTEGRATED 
PLANNER  AND  EXECUTOR 


Highest  level  of  planning  and  integration 
Strategic  Level  (Joint  Staff,  GCC,  etc.) 

Works  with  Joint  and  Combined  Logistics,  Planning,  and  Ops 
Links  operation  strategy  to  contract  integration  in  OPLANs 
High-level  civilians  and  senior-grade  officers 
Liaison  functions  with  broader  stakeholders  -  NGO  and  PVO 
Designs  and  exercises  contracting  support  plans 
Comprehensive  analysis  to  create  contract  schema 
Develops  Annex  W  (CSIP) 

Standardized  training  and 
education  essential: 

-  protocols 

-  ethics 

-  control  and  oversight 

-  most  experienced 

-  highest  education 

-  joint  and  multidisciplinary  experience 


Source.  (Yoder,  2010) 

Note.  CSIP  =  Contract  Support  Integration  Plan 


An  IPE  must  be  strategically  positioned  within  the  organization  to 
achieve  the  highest  levels  of  integrative  planning.  The  IPE’s  primary 
mission  is  creating  and  validating  a  comprehensive  Annex  W  to  comple¬ 
ment  all  elements  of  the  OPLAN.  Ideally,  the  IPE  position  should  be 
placed  within  the  Joint  Staff,  at  GCC,  and  at  the  highest  operational  and 
planning  staffs  within  each  Service  branch. 

The  IPE  will  create  and  validate  Annex  W  in  all  key  GCC  OPLAN 
and  concept  plan  (CONPLAN)  elements.  (Specific  content  elements  of 
Annex  W  are  presented  later  in  this  article.)  Because  of  the  complexity 
and  magnitude  of  the  tasks  involved  in  creating  and  validating  com¬ 
prehensive  plans,  the  IPE  requires  a  supporting  staff  and  subordinate 
expertise  in  key  strategic  and  analytical  areas,  such  as  OPLAN  analysis, 
logistics  assessments,  contracting,  and  similar  professional  disciplines. 

Of  note,  most  organizations  do  not  have  a  dedicated  contracting  IPE 
(by  any  moniker)  within  their  organizational  structure.  Traditionally, 
the  joint  logistics  (J-4)  organizations  have  embedded  contracting  offi¬ 
cers.  However,  the  contracting  positions  within  J-4,  or  within  traditional 
logistics  organizations,  have  been  utilized  as  adjunct  positions  to  the 
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broader  logistics  functional  planning.  Additionally,  because  of  the  rela¬ 
tively  low  military  rank  and  lack  of  seniority,  contracting  personnel 
on  J-4  staffs  often  lack  both  the  credentials  and  the  clout  to  effectively 
execute  the  requirements  proposed  for  the  IPE. 

Despite  the  DoD  Components  and  military  services  lacking  an  IPE 
at  the  strategic  level,  the  National  Defense  Authorization  Act  of  2008 
(NDAA,  2008)  made  significant  impact  at  addressing  credentialed  per¬ 
sonnel  shortfalls  at  the  strategic  level.  The  NDAA  2008  authorized  and 
established  the  Joint  Contingency  Acquisition  Support  Office  (JCASO), 
directed  by  a  military  one-star  flag  officer,  positioned  within  the  Defense 
Logistics  Agency.  JCASO  has  a  staff  of  49  personnel  expressly  to  provide 
IPE  strategic-level  assistance  and  contract  support  to  GCCs.  These 
JCASO  specialists  work  with  GCC  planning  staffs  to  incorporate  essen¬ 
tial  contracting  plans  at  the  GCC.  According  to  Navy  Rear  Admiral  Ron 
J.  MacLaren,  director  of  JCASO,  each  GCC  is  allocated  two  special¬ 
ists  from  JCASO  to  assist  in  the  development  and  exercise  of  each  key 
OPLAN’s  Annex  W  (MacLaren,  2012). 

Will  the  DoD  Components  and  the  military  services  embrace  the 
TTM,  particularly  the  IPE  function  established  by  NDAA  2008  as  the 
JCASO?  Currently,  JCASO  has  not  been  empowered  with  authority  to 
compel  GCC  or  the  DoD  Components  and  military  services  to  utilize 
their  OCS  development  functions.  Rather,  it  represents  an  advisory  group 
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that  must  “sell”  its  capabilities  to  improve  mission  support  through  inte¬ 
grative  planning  (MacLaren,  2012).  Only  time  and  sound  metric  analysis 
will  prove  whether  or  not  the  JCASO  is  effective  at  creating  the  needed 
Annex  W  OCS  plans  mandated  and  needed  for  key  GCC  OPLANs. 

What  specifically  will  the  IPE  position  accomplish,  and  what  will  it 
achieve?  If  the  warfighters  are  to  embrace  OCS,  they  must  understand 
what  essential  functions  the  IPE  achieves,  and  how  those  functions  will 
yield  benefits. 

Phase  Zero— Planning,  Exercising,  and  Rehearsal 

Phase  Zero,  generally  known  in  GCC  planning  arenas  as  the  shaping 
phase,  is  adopted  by  the  OCS  contracting  community  as  the  planning 
and  exercising  phase.  Traditional  military  jargon  defines  Phase  Zero 
as  “shaping.”  Phase  Zero  contracting  in  the  integrative  strategic  plan¬ 
ning  arena  is  the  advance  planning,  exercising,  and  rehearsal  of  robust 
contracting  support  plans  designed  to  complement  the  GCC’s  deliberate 
and  contingency  planning  process.  Realistically,  the  contracting  com¬ 
munity  and  the  warfighter  have  the  same  vision  for  Phase  Zero— get  the 
plans  in  place,  then  rehearse,  validate,  and  update  them  to  reflect  current 
realities.  In  essence,  Phase  Zero  contract  planning  and  the  creation  of 
OPLAN  Annex  W  became  mandatory  under  NDAA  2008  (Government 
Accountability  Office  [GAO],  2011).  The  authorization  and  supporting 
guidance  under  JP  4-10  (C  JCS,  2008),  requires  all  GCCs  to  create  Annex 
W  for  OPLANs,  representing  the  embodiment  of  Phase  Zero  integrative 
planning.  However,  despite  the  mandate,  what  is  particularly  disconcert¬ 
ing  is  that  the  GAO  recently  determined  that  only  four  out  of  39  OPLANs 
requiring  comprehensive  Annex  W  integration  plans  actually  had  them 
(GAO,  2011).  The  low  rate  of  Annex  W  integration  may  be  a  result  of 
the  challenges  in  assimilation  and  normalization  of  new  doctrine  and 
processes  that  DoD  initially  approved  in  2008.  MacLaren  indicates  that 
significant  work  is  ahead  to  get  all  the  GCC  OPLAN  Annex  W  support 
plans  in  place  and  exercised  (MacLaren,  2012).  The  authors  contend 
that  current  operational  tempos,  along  with  constrained  budgets,  may 
preclude  achieving  fully  integrated  exercises  and  rehearsals  for  all 
OPLANs,  as  these  rehearsals  can  carry  a  huge  price  tag.  However,  failure 
to  exercise  and  rehearse,  based  on  recent  and  well-documented  problems 
in  Iraq  and  Afghanistan,  results  in  costs  that  far  outweigh  the  up-front 
costs  to  fully  vet  Annex  W  plans.  Deliberate  planning  and  contingency 
planning  are  different— the  first  is  not  necessarily  time-sensitive,  but 
the  second  may  be  very  time-sensitive  and  is  often  constrained.  While 
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JCASO  has  assigned  two  specialists  at  each  GCC  to  assist  in  creating 
and  exercising  each  Annex  W,  ultimately,  the  DoD  Component  and  mili¬ 
tary  services’  contract  warrant  holders  will  be  responsible  for  providing 
actual  contract  support,  and  must  be  included  in  the  planning,  exercise, 
rehearsal,  and  execution  of  the  OPLAN.  For  the  most  critical  OPLANs, 
sound  strategy  requires  the  exercise  and  rehearsal  of  each  Annex  W  with 
the  personnel  that  will  ultimately  be  called  into  action. 

Ideally,  each  OPLAN  and  CONPLAN  will  have  an  Annex  W,  fully 
drafted,  exercised,  rehearsed,  analyzed,  and  revised.  The  doctrinal 
framework  published  in  JP  5-0,  along  with  JP  4-10,  is  key  for  the  design 
and  integration  of  contracting  into  OPLANs.  The  authors  note  that  JP 
5-0  does  not  currently  require  an  Annex  W—  only  JP  4-10  requires  it.  The 
objective  is  to  embed  and  synchronize  the  OCS  plan  with  all  elements 
of  the  OPLAN  to  meet  the  commander’s  intent.  Properly  constructed 
Annex  W  plans  must  include  elements  such  as,  but  not  limited  to,  per¬ 
sonnel/organizational  structures  and  authorities;  business  protocols, 
including  special  statutory  and  regulatory  provisions  under  declared 
contingencies;  scheme  of  operations;  synchronization  with  the  battle 
plan;  oversight;  management  and  auditing;  personnel  regulations  and 
provisions;  spend  analysis  integration;  synchronization  with  broader 
strategic  objectives;  and  metrics  for  assessment  of  the  efficiencies  and 
effectiveness  of  embedded  plans  and  actions  (Yoder,  2011). 

To  ensure  the  efficacy  of  the  integrated  Annex  W  plan,  the  IPE 
must  act  as  a  strategic  liaison  with  key  stakeholders.  Analytical  assess¬ 
ments  of  the  Annex  W  plan  may  utilize  strength,  weakness,  opportunity, 
threat  (SWOT)  and  capability  gap  analysis  techniques.  The  SWOT 
method  allows  the  IPE  to  evaluate  the  strengths,  weaknesses/limita¬ 
tions,  opportunities,  and  threats;  and,  ultimately,  the  potential  efficacy 
of  the  OPLAN’s  integrated  contracting  plan.  The  capability  gap  analysis 
determines  the  support  and  provisioning  gaps  in  the  OPLAN  that  may 
be  addressed  through  contracted  support. 

Contracting  Phases— Complementing  Warfighter  Strategy 

Contingency  contracting  planning  must  complement  and  seam¬ 
lessly  integrate  with  the  DoD  and  Combatant  Command  APEX  planning 
process.  On  the  surface,  the  two  processes  appear  distinctly  different, 
with  phasing  graphics  exhibiting  dissimilar  phasing  models  and  activ¬ 
ity  descriptions  within  each  phase.  Examination  of  each  model  reveals 
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that  the  two  processes  are  complementary,  but  care  must  be  taken  to 
ensure  that  contracting  phasing  supports  and  is  parallel  with  opera¬ 
tional  planning. 

The  DoD  deliberate  planning  process  generally  includes  six  phases, 
although  the  number  and  types  of  phases  are  contingent  upon  the  char¬ 
acteristics  of  the  joint  operation.  For  instance,  a  combat  operation  will 
be  phased  differently  than  a  humanitarian  relief  operation.  JP  5-0 
describes  the  notional  phasing  construct  (Figure  2)  as  follows:  Phase  0 
(Shape)  includes  normal  and  routine  military  activities  as  well  as  secu¬ 
rity  cooperation  activities  that  are  contained  within  the  theater 
campaign  plan  (TCP).  The  TCP  includes  steady  state  operations  and 
activities  intended  to  promote  international  legitimacy  and  cooperation 
with  friends  and  allies,  while  dissuading  adversaries.  Phase  I  (Deter) 
includes  those  activities  that  demonstrate  “joint  force  capabilities  and 
resolve”  in  response  to  an  adversary’s  undesirable  actions.  Actions 


FIGURE  2.  WARFIGHTER’S  NOTIONAL  OPERATIONAL  PLAN  PHASES 


Notional  Operation  Plan  Phases 


OPLAN  OPORD  OPORD  Execution  OPORD 
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Source.  (CJCS,  2011) 
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include  preparation  for  deployment,  deployment,  and  shows  of  force 
designed  to  influence  an  adversary’s  decision-making  process.  Phase  II 
(Seize  Initiative)  begins  the  “application  of  appropriate  joint  force  capa¬ 
bilities”  to  “delay,  impede,  or  halt  an  adversary’s  initial  aggression.”  This 
phase  sets  the  conditions  for  the  successful  implementation  of  the  Phase 
III  (Dominate)  phase.  Phase  III  includes  actions  designed  to  “break  the 
enemy’s  will ...  or,  in  noncombat  operations,  to  control  the  operational 
environment.”  Phase  IV  (Stabilize)  is  “required  when  there  is  no  fully 
functional  legitimate  civil  governing  authority,”  and  joint  forces  must 
perform  limited  local  governance  and  other  activities  to  allow  for  a  res¬ 
toration  of  stability  and  a  return  to  normalcy.  This  phase  may  require 
joint  force  cooperation  and  coordination  with  intergovernmental  orga¬ 
nizations,  nongovernmental  organizations,  or  other  civilian  agencies. 
Phase  V  (Enable  Civil  Authority)  includes  the  provision  of  “joint  force 
support  to  legitimate  civil  governance”  in  theater  as  well  as  assistance 
with  the  provision  of  essential  services  to  local  populations.  It  usually 
includes  redeployment  operations,  especially  of  combat  forces,  as  well 
as  the  planning  for  transition  back  to  Phase  0  or  steady-state  operations. 
Figure  3  illustrates  the  notional  operation  plan  phases  (C  JCS,  2011,  pp. 
III-41-III-44). 


FIGURE  3.  THREE-TIER  MODEL:  TIER  THREE,  INTEGRATED 
PLANNER  AND  EXECUTOR 
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FIGURE  4.  CONTRACTING  PHASE  ZERO:  PLAN,  EXERCISE,  REHEARSE,  AND  SYNCHRONIZE 
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Since  NDAA  2008,  contracting  has  utilized  five  support  phases. 
Phase  Zero  (Figure  4)  is  the  planning,  exercise,  and  rehearsal  phase. 
During  this  phase,  contingency  contracting  planners  work  with  com¬ 
batant  command  staffs  in  the  deliberate  planning  process  to  develop 
the  Annex  W  for  each  campaign  and  operations  plan.  Exercising  and 
rehearsing  these  plans  is  imperative  to  ensure  they  meet  the  warfighter’s 
expectations  and  correspond  with  Phase  0  of  the  deliberate  planning 
process.  Phase  I  is  deployment,  during  which  initial  contracting  opera¬ 
tions  and  relationships  are  established,  especially  to  provide  basic  life 
support  requirements  for  arriving  personnel.  It  corresponds  roughly 
with  the  first  half  of  the  warfighter’s  Phase  I  (Deter).  The  contracting 
Phase  II  is  joint  reception,  staging,  onward  movement,  and  integration. 
This  phase  includes  the  arrival  of  the  main  body  of  deploying  forces 
and  their  equipment.  It  requires  the  establishment  of  more  robust  con¬ 
tracting  initiatives,  including  expanded  life  support  requirements  and 
temporary  construction  to  support  the  deployed  force  and  corresponds 
with  the  warfighter’s  Phase  I  (Deter),  as  well  as  elements  of  Phase  II 
(Seize  Initiative).  Phase  III  (Sustainment)  provides  contracting  support 
from  the  completion  of  the  build-up  phase  until  the  beginning  of  the 
redeployment  of  the  force.  It  also  includes  stability  and  reconstruction. 
This  contingency  contracting  phase  corresponds  with  the  warfighter’s 
Phase  II  (Seize  Initiative),  Phase  III  (Dominate),  Phase  IV  (Stabilize), 
and  portions  of  Phase  V  (Enable  Civil  Authority).  The  contracting  Phase 
IV  is  termination  and  redeployment,  and  includes  activities  that  support 
the  “pressure  and  urgency  to  send  the  deployed  forces  home.”  It  also 
includes  close-out  of  existing  contracts  as  well  as  establishing  contracts 
for  follow-on  forces,  such  as  United  Nations  peacekeepers  (Defense 
Procurement,  2012,  pp.  111-121). 

DoD  Directive  (DoDD)  3020.49  mandates  the  coordination  and 
synchronization  of  contracting  with  broader  warfighter  OPLANs  (DoD, 
2009).  As  Figure  4  illustrates,  contracting  phasing  does  not  correspond 
exactly  with  the  warfighter’s  phasing  plan.  This  lack  of  correspondence 
can  lead  to  misunderstanding,  lack  of  communication  within  the  plan¬ 
ning  staff,  and  a  failure  of  coordination  and  synchronization.  The  authors 
strongly  suggest  that  contracting  personnel  revise  the  phasing  plan  to 
more  closely  correspond  with  the  JP  5.0  phasing  construct. 
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Phase  Zero  and  Mandatory  Pillars  for  Strategic 
Contracting  Integration 

As  defined  previously.  Phase  Zero  is  the  planning,  exercising,  and 
rehearsal  phase  of  military  operations— properly  establishing  and  vet¬ 
ting  the  contracting  plan  prior  to  an  actual  event  or  crisis.  To  function 
effectively  within  the  established  and  existing  military  deliberate  and 
contingency  planning  framework,  the  IPE  and  associated  functions 
must  be  designed  within  three  main  pillars:  personnel,  platforms,  and 
protocols  (Figure  5).  Failure  to  integrate  contracting  with  all  of  the  three 
primary  pillars  will  result  in  suboptimization  or  outright  contract  sup¬ 
port  and/or  mission  failure  (Yoder,  2010). 


FIGURE  5.  MANDATORY  PILLARS  FOR  INTEGRATIVE  SUCCESS 
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Note.  APEX  =  Adaptive  Planning  &  Execution  System;  GCC  =  Global  Combatant 
Commander;  IPE  =  Integrated  Planner  and  Executor;  JCASO  =  Joint  Contingency 
Acquisition  Support  Office;  PD2  =  Procurement  Defense  Desktop;  SPS  =  Standard 
Procurement  System;  TPFDD  =  Time  Phased  Force  Deployment  Data 


The  first  pillar,  personnel,  should  be  addressed  by  implementing 
the  TTM  and  particularly  the  IPE.  The  second  pillar,  platforms,  is 
addressed  by  integrating  contracting  throughout  all  phases  of  military 
operations  and  into  the  existing  warfighters’  platform  for  planning  and 
execution— the  APEX.  Additionally,  it  must  be  embedded  with  other 
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APEX-complementary  platforms,  such  as  the  Time  Phased  Force  and 
Deployment  Data  (TPFDD)  system.  The  third  pillar,  protocols,  repre¬ 
sents  the  existing  or  desirable  set  of  rules  and  procedures,  including 
sound  business,  planning,  and  military  doctrine  that  govern  the  planning 
and  execution  of  the  contracting  plan  within  the  broader  OPL  AN.  Figure 
5  highlights  the  three  pillars  and  associated  elements. 

Protocols  include,  but  are  not  limited  to,  the  strategic  planning 
guidance  established  by  the  GCC;  strategic  purchasing  guidance  and 
mandates;  JP  4-10,  JP  5-0,  JP  4-0,  Doctrine  for  Logistic  Support  of  Joint 
Operations  (C  JCS,  2000),  and  other  doctrinal  publications;  and  associ¬ 
ated  mandates  for  constructing  and  implementing  Annex  W  for  each 
unique  OPLAN.  Additionally,  acquisition-  and  contracting-specific  laws, 
regulations,  and  guidance  must  be  utilized  including,  but  not  limited 
to,  the  Federal  Acquisition  Regulation  (2012)  and  the  Defense  Federal 
Acquisition  Regulation  Supplement  (Defense  Procurement,  2013),  as 
well  as  any  specific  military  service  acquisition  regulations. 


Annex  W  must  include  all  of  the  key  elements  for 
mission  success  and  address  the  three  mandatory 
pillars  for  integrative  success:  personnel, 
platforms,  and  protocols. 


The  Integrated  Planner  and  Executor  within  Strategic 
Planning,  APEX  Products,  and  Annex  W 

Joint  strategic  planning  products  include,  but  are  not  limited  to,  GCC 
estimates,  base  plans,  OPLANs,  CONPLANs,  warning  orders,  planning 
orders,  alert  orders,  operation  orders  (OPORD),  execute  orders,  fragmen¬ 
tary  orders,  and  deployment  orders,  along  with  all  annexes  including  the 
newly  mandated  Annex  W.  These  products  are  alien  to  most  contracting 
and  acquisition  professionals  because,  traditionally,  contracting  and 
acquisition  personnel  have  not  played  a  key  role  in  the  production  or 
management  of  these  critical  documents.  The  GAO  recently  conducted 
an  audit  of  39  OPLANs  requiring  an  integrated  Annex  W,  and  found  that 
only  four  operational  contracting  plans  had  been  produced  (GAO,  2011). 
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FIGURE  6.  MINIMUM  ELEMENTS  INCLUDED  IN  AN  INTEGRATED  ANNEX  W 
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2. 

Primary  and  secondary  customers 

3. 

Anticipated  requirements  (in  relative  time-phase) 

4. 

Forces  deploying  in  sequence  and  duration 

5. 

Operational  locations 

6. 

Lead  service 

7. 

Organization  structure  (Head  of  Contracting  Activity,  Joint  Acquisition 

Review  Board,  etc.) 

8. 

Supported  and  supporting  relationships 

9. 

Command  and  control  relationships 

10. 

Procedures  for  appointing,  training,  and  employing  field  ordering  officers, 
contacting  officer  representatives,  disbursing  agents,  and  government 
purchase  card  holders 

11. 

Procedures  for  defining,  validating,  processing,  and  satisfying  customer 
requirements 

12. 

Procedures  for  budgeting  receipt  of  supplies/services  and  payments  to 
vendors 

13. 

Procedures  for  closing  out  contracting  operations  and  redeployment 

14. 

Supplies  and  services  anticipated  locally,  local  customs,  laws,  taxes,  SOFAs, 
host  nation  support,  Acquisition  Cross-Service  Agreements,  vendor  base,  etc. 

15. 

Infrastructure,  office  location,  security  measures,  kits,  etc. 

16. 

Security  requirements  and  procedures  for  contracting  and  contractor 
personnel 

17. 

Standards  of  support— processing  times,  turn-around-time,  Procurement 
Acquisition  Lead  Time,  and  reporting 

18. 

Specific  statutory/regulatory  constraints  or  exemptions,  special  authorities, 
and  programs 

19. 

Relief  in  place/transfer  of  authority 

20. 

Contractor  restrictions  (movement,  basing,  etc.,  time-phase  specific) 

21. 

Guidance  on  transferring  Logistics  Civil  Augmentation  Program  support  to 
theater  support  contracts  by  function  and/or  phase  of  the  operation 

22. 

Special  authorities  and  programs  (Commanders’  Emergency  Response 
Program-Counterinsurgency) 

23. 

Postcontract  award  actions  (management,  closeout,  de-obligation,  etc.) 

24. 

Contractor  support,  civil  augmentation  programs 

25. 

Mandated  solicitation  and  contract  provisions 

26. 

Human  trafficking  mandates,  indemnity,  and  Military  Extraterritorial 
Jurisdiction  Act  provisions 

Source:  (Yoder,  2010) 
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Clearly,  given  the  defined  content  of  Annex  W,  the  contracting  at  the 
strategic  IPE  level  must  be  included  in  all  phases  of  planning  and  in  the 
production  of  key  APEX  products.  Annex  W  must  include  all  of  the  key 
elements  for  mission  success  and  address  the  three  mandatory  pillars  for 
integrative  success:  personnel,  platforms,  and  protocols.  The  integrated 
Annex  W  must  include,  at  a  minimum,  those  elements  deemed  essential 
for  mission  accomplishment,  while  addressing  cost  and  affordability 
within  the  overall  OPLAN.  The  contents  include,  but  are  not  limited  to, 
as  indicated  in  Yoder  (2010),  the  26  elements  shown  in  Figure  6. 

Without  a  comprehensive  planning  capability,  most  missions  will  be 
negatively  affected.  Clearly,  the  IPE,  properly  positioned  within  the  plan¬ 
ning  community,  can  better  create  and  assess  the  Annex  W  capabilities 
within  the  three  main  pillars— personnel,  platforms,  and  protocols— to 
allow  for  future  success. 


Conclusions 

To  date,  contracting  has  not  been  fully  integrated  into  military 
planning  and  execution.  Some  significant  strides  have  been  made  to 
better  assimilate  contracting  at  the  strategic  level,  including  Dr.  Jacques 
Gansler’s  (2007)  report,  Urgent  Reform  Required,  and  the  recently  pub¬ 
lished  doctrine  contained  in  JP  4-10  (C  JCS,  2008).  However,  despite  the 
push  toward  better  integration,  including  the  newly  formed  JCASO,  the 
DoD  still  lacks  a  manifest  comprehensive  planning  and  executing  capa¬ 
bility,  as  evidenced  most  recently  in  the  final  report  of  the  Commission 
on  Wartime  Contracting  in  Iraq  and  Afghanistan  (2011). 

The  lack  of  planning  and  sound  contract  integration  at  the  strate¬ 
gic  level  leads  to  loss  of  efficiencies,  lack  of  effectiveness,  and,  in  many 
cases,  outright  fraud  of  the  executing  participants  as  highlighted  in 
the  2011  report  of  the  Commission  on  Wartime  Contracting  in  Iraq 
and  Afghanistan. 

The  functions  of  the  IPE  and  mandates  for  OCS  (C  JCS,  2008),  includ¬ 
ing  generating  a  thoroughly  vetted  Annex  W,  are  so  massive  that  the 
Services  have  recently  contracted  out,  or  outsourced,  some  of  the  require¬ 
ment  (Yoder,  2011).  However,  outsourcing  this  critical  function  may  only 
make  matters  worse,  in  that  key  decisions  will  be  left  in  the  purview  of 
nongovernment  personnel,  including  decisions  of  further  contracting, 
along  with  other  possible  conflicts  of  interest  and  potential  for  corruption. 
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The  authors  contend  that  the  best  means  to  accomplish  integration 
into  existing  war  planning  systems  is  by  congressionally  mandating, 
authorizing,  and  funding  (via  appropriation)  the  IPE  positions  at  the 
flag  and  senior  executive  service  levels  within  Service  structures.  In  the 
short  term,  the  authors  recommend  that  JCASO  have  more  status  and 
capability  within  GCC  and  Service  staff's,  particularly  in  assisting  the 
GCC  staff  to  establish,  monitor,  and  manage  Annex  W  within  the  APEX 
framework.  This  will  require  greater  engagement  capability  than  cur¬ 
rently  exists.  In  the  long-term,  the  authors  recommend  Congressional 
approval  and  funding  of  IPE  positions  organically  within  the  GCC  staff, 
providing  them  with  direct  authority  for  the  development,  review,  and 
employment  of  Annex  W.  This  greater  presence  and  authority  at  the  IPE 
level  within  the  GCC  staff  represents  the  level  of  bona  fide  commitment 
to  solve  a  long-standing  problem  that,  without  correction,  will  continue 
to  fester  and  plague  Service  chiefs,  military  commanders,  Congress,  and 
taxpayers.  Additionally,  fully  aligning  and  integrating  the  contracting 
community’s  processes  with  the  joint  community  planning  system  is 
imperative.  Failure  to  do  so  may  result  in  lack  of  communication,  lack 
of  synchronization  of  support  plans,  and  marginalization  of  contracting 
personnel  within  the  GCC  planning  staff.  Implementing  PZCO  planning 
through  sound  public  policy,  congressional  authorization  and  funding, 
and  the  Services’  commitment  to  fully  integrate  contracting  within 
the  three  pillars— personnel,  platforms,  and  protocols— is  the  proactive 
move  toward  success. 
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The  Challenges  in  Meeting 
OSD’s  Obligation  and 
Expenditure  Rate  Goals: 

A  Closer  Look  at  Potential 
Causal  Factors, 

Their  Groupings, 
and  How  They  Modulate 


f  Co/  Robert  L.  Tremaine,  USAF  (Ret.), 
and  Donna  J.  Kinnear-Seligman 


Managing  an  acquisition  program  in  the  DoD  is  a 
complicated  process.  The  turbulence  created  by  funding 
instability  can  make  it  even  more  difficult.  Nonetheless, 
to  help  program  offices  maintain  their  overall  funding 
execution  pace,  the  Office  of  the  Secretary  of  Defense 
(OSD)  instituted  Obligation  and  Expenditure  rate  goals 
over  two  decades  ago.  For  numerous  reasons,  acquisi¬ 
tion  program  managers  have  found  it  difficult  to  meet 
established  Obligation  and  Expenditure  rate  goals.  For 
purposes  of  this  article,  and  based  on  Defense  Acquisi¬ 
tion  University  and  OSD  subject  matter  expertise,  the 
authors  looked  more  closely  at  the  potential  causal 
factors  that  could  be  interfering  with  the  achievement 
of  these  goals. 
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Several  months  ago.  Dr.  Nancy  Spruill,  director  of  Acquisition 
Resources  and  Analysis,  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics,  solicited  support  from  the  Defense 
Acquisition  University  (DAU)  to  help  uncover  the  causal  factors  that  could 
be  interfering  with  the  attainment  of  OSD’s  Obligation  and  Expenditure 
rate  goals.  To  learn  more  about  the  intervening  obstacles,  DAU,  with 
assistance  from  OSD,  developed  a  comprehensive  survey  that  queried 
experienced  and  high-level  DoD  personnel  involved  in  a  weapon  pro¬ 
gram’s  decision  chain.  The  data  might  also  indicate  the  prevalence  of  any 
significant  variances  among  the  factors  that  could  be  undermining  pro¬ 
gram  execution  itself.  Results  of  the  study  (Higbee,  Tremaine,  Seligman, 
&  Arwood,  2013)  were  presented  to  Assistant  Secretary  of  Defense  for 
Acquisition  Katrina  McFarland  and  other  senior  OSD  personnel. 

Research  Methodology 

Two  hundred  and  twenty-nine  DoD  personnel  responded  to  this 
survey.  The  respondents  were  comprised  of  program  office  personnel 
(program  managers,  deputy  program  managers,  budget  and  financial 
managers,  and  contracting  officers);  program  executive  officers  and  their 
chief  financial  officers;  and  a  variety  of  senior  staff  at  OSD  including 
Headquarters  Financial  Management  senior  staff  and  Senior  Acquisition 
Executive  (SAE)  staff  (Table  1).  Because  several  functional  areas 
reflected  lower  response  rates,  a  more  detailed  analysis  of  the  causal 
factors  was  restricted  to  an  aggregate  sample  size  given  the  confidence 
levels  required  to  draw  any  inferences  or  conclusions. 

TABLE  1.  INDIVIDUAL  RESPONDENT  GROUPS 


Survey  Respondent  Details 


ACAT  Levels 

Respondent  Groups 

Totals 

Respondent 

Program 

Senior 

Response 

Distribution3 

1  II 

III 

Officeb 

PEOc 

Staffd 

Responses 

Queried 

Rate 

Total 

91  28 

23 

142 

63 

24 

229 

698 

33% 

a  Includes  sampling  from  all  DoD  Components  and  several  Defense  Agencies 

b  Program  managers,  deputy  program  managers,  business-financial  management  (BFM)  managers, 
deputy  BFM  managers,  and  contracting  officers 
c  Program  executive  officers  (PEO),  deputy  PEOs,  and  their  chief  financial  officers 
d  Headquarters,  Financial  Management  and  Senior  Acquisition  Executive  staff 
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Respondents  ranked  the  impact  of  64  factors  under  nine  categories 
(Figure  1).  The  researchers  then  assessed  the  rankings  using  atop  box 
(TB)  three  methodology  (i.e.,  the  percentage  of  5, 6,  and  7  responses  on  a 
Likert-like  scale  from  1-7).  Since  the  frequency  of  occurrence  for  some 
factors  could  also  be  contributing  to  the  interference,  the  researchers 
included  an  additional  selection  (e.g.,  daily,  weekly,  monthly,  etc.)  to 
isolate  any  potential  ignition  areas  for  any  factor. 


FIGURE  1.  FACTOR  CATEGORIES 
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Discussion 


Factor  Distribution 

Figure  2  shows  the  distribution  of  all  64  factors  assessed.  Three 
factors  reported  an  impact  rating  of  two  standard  deviations  above 
the  mean  (denoted  by  +2er);  six  factors  reported  an  impact  rating  of  one 
standard  deviation  above  the  mean  (denoted  by  +ler);  and  22  factors  rose 
above  an  average  impact  rating  (denoted  by  x).  The  remaining  33  factors 
fell  below  x. 

FIGURE  2.  RESPONDENT  HISTOGRAM 


IMPACT  FACTOR  DISTRIBUTION 


Impact  Ratings 


Nineteen  of  the  22  factors  measured  for  frequency  of  occurrence 
resulted  in  an  impact  rating  above  39  percent.  Sometimes  Just  one  occur¬ 
rence  of  that  factor  appeared  to  have  a  significant  impact. 

Causal  Factors  Rank  Ordered 

Table  2  lists  the  relative  ranking  of  all  64  factors  in  the  context  of 
TB  in  descending  order.  This  ranking  provides  a  comprehensive  view  of 
all  factors  although  the  remaining  discussion  in  this  article  addresses 
only  the  factors  above  x.  One  particular  factor,  “Unrealistic,  overly 
optimistic  spend  plans”  (F10),  is  important  to  note  since  it  serves  as  a 
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written  forecast  of  a  program’s  funding  needs  and  initially  establishes 
Obligation  and  Expenditure  projections.  However,  spend  plans  are  also 
subjected  to  so  many  real  world  eventualities  that  updating  them  can 
become  a  full-time  job. 

Factors  and  Respondent  Groups 

Figure  3  accounts  for  the  31  factors  above  the  mean  and  by  respon¬ 
dent  group  as  depicted  in  Table  1.  The  31  factors  were  the  only  ones 
further  evaluated  in  this  study  unless  a  factor  shifted  above  x  after  a 
more  detailed  correlation  delineation  (e.g.,  Acquisition  Category  [ACAT]) 
levels,  military  components,  position,  etc.).  Unexpectedly,  the  individual 
factors  showed  widespread  perception  disparities  among  the  respondent 
groups  for  the  factors  that  fell  below  +2er.  After  analyzing  the  specific 
individual  factors  among  all  the  respondent  groups,  seven  of  the  31  fac¬ 
tors  had  an  unusually  large  a.  As  a  result  of  these  conspicuous  gaps,  the 
authors  turned  to  the  qualitative  data  and  watched  for  any  strong  cor¬ 
relations  (e.g.,  positive  quantitative  correlation  coefficients  (r)  >  0.7)  to 
better  understand  the  reasons  for  the  differences  as  well  as  the  influence 
of  any  intervening  and/or  moderating  factor  couplings.  The  remaining 
discussion  addresses  the  31  impact  factors  in  descending  order  from 
highest  to  lowest. 

Factors  Ranked  Two  Standard  Deviations  Above  the 
Mean  (+  2d) 

In  Figure  3,  Late  release  of  full  obligation/budget  authority  due  to 
Continuing  Resolution  Authority  (CRA)  (F1),  Contract  negotiations’ 
delays  (F2),  and  Contract  award  delays  (F3)  all  rose  above  2er  where  67 
percent  or  more  of  the  respondents  claimed  they  had  the  highest  adverse 
impact  of  all  factors  measured.  The  occurrence  of  CRA  had  the  most 
significant  negative  impact  to  Obligation  and  Expenditure  rates.  It  also 
had  one  of  the  smallest  variances  (a)  among  the  respondent  groups.  Even 
with  the  expectation  that  CRA  might  prevail  and  the  subsequent  plan¬ 
ning  that  followed  for  such  a  likely  event,  many  PMs  pointed  to  an  overly 
conservative  and  slow  internal  vetting  process  posture  that  created  addi¬ 
tional  obstacles  in  meeting  OSD  goals.  In  their  responses  to  qualitative 
questions,  several  PMs  recommended  using  some  sort  of  “CRA  variable” 
to  temporarily  offset  the  consequences  of  CRA  if  the  required  funds  were 
not  released  as  originally  projected.  Next  in  rank  order  were  contract 
negotiations  and  contract  award  delays.  The  respondents  emphasized 
that  DoD  could  fix  the  problem  more  readily  since  unlike  CRA,  these 
factors  were  under  internal  control.  When  asked  what  could  be  done 
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TABLE  2.  IMPACT  FACTOR  RATINGS  IN  AGGREGATE  DESCENDING  ORDER 


Factors  Rated  by  Adverse  Impact 

TB 

X 

a 

F1 

Late  release  of  full  obligation  budget  authority  due  to  Continuing 
Resolution  Authority  (CRA) 

69% 

5.29 

2.41 

F2 

Contract  negotiation  delays 

67% 

5.06 

2.59 

F3 

Contract  award  delays 

67% 

5.00 

2.56 

F4 

Shortage  of  contracting  officers 

64% 

4.79 

2.58 

F5 

Congressional  mark/rescission 

61% 

4.87 

2.65 

F6 

Contractor  proposal  prep  delays 

60% 

4.87 

2.59 

F7 

OSD-directed  Resource  Management  Decision  (RMD) 

58% 

4.50 

2.63 

FB 

Request  for  Proposal  (RFP)  prep  delays 

57% 

4.63 

2.46 

F9 

Source  selection  delays 

55% 

4.44 

2.53 

F!0 

Unrealistic,  overly  optimistic  spend  plans 

52% 

4.30 

2.44 

F" 

Changes  in  user  requirements 

51% 

4.16 

2.43 

F'2 

Changes  to  program  acquisition  strategy 

51% 

4.41 

2.52 

F'3 

Changes  in  other  stakeholder  requirements 

50% 

4.32 

2.34 

F'4 

Preparing  Defense  Acquisition  Executive  (DAE)-level  review 
and  decision 

50% 

4.15 

2.18 

F’5 

Lack  of  decision  authority  at  expected  levels 

50% 

4.22 

2.52 

F’6 

Implementation  of  new  OSD/Service  policy 

49% 

4.20 

2.59 

F’7 

Component-directed  Program  Objective  Memorandum 
(POM)  adjustment 

49% 

4.26 

2.51 

F18 

Awaiting  reprogramming  action 

49% 

4.23 

2.44 

F’9 

Changes  in  user  priorities 

47% 

4.00 

2.38 

p20 

Realistic  spend  plans,  but  risks  materialized 

45% 

4.00 

2.21 

F2' 

Program  delays  from  additional  development,  testing,  or 
other  prerequisite  events 

44% 

4.09 

2.35 

p22 

Defense  Contract  Audit  Agency  (DCAA)  administrative  actions 

44% 

3.92 

2.61 

F23 

Unplanned  Congressional  adds  to  Program  Baseline  (PB)  request 

43% 

3.90 

2.41 

F24 

Use  of  undefinitized  contract  action  delays 

42% 

3.73 

2.56 

F25 

Expenditure  contingent  on  hardware  delivery 

41% 

3.92 

2.41 

F26 

Loss  of  funding  through  reprogramming  action  to  higher  priority 
requirements  to  program  executive  officer  (PEO)  portfolio 

41% 

3.89 

2.46 

F27 

Lack  of  experience  levels  in  key  acquisition  functional  areas 

40% 

3.90 

2.30 

p28 

Awaiting  DAE-level  review  and  decision 

40% 

3.50 

2.42 

p29 

Shortage  of  cost  estimators 

40% 

3.67 

2.37 

p30 

Shortage  of  business/finance  personnel 

39% 

3.66 

2.32 

F31 

Programmatic  conflicts  between  government  and 
prime  contractor 

39% 

3.66 

2.32 
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Factors  Rated  by  Adverse  Impact  Continued 

TB 

X 

a 

R32 

Preparing  Service  Acquisition  Executive/Component 

Acquisition  Executive  (SAE/CAE)-level  review  and  decision 

38% 

3.74 

2.02 

p33 

Delays  in  contractor  payment  due  to  late  invoices 

37% 

3.67 

2.35 

F34 

Unobligated  prior  year  funding  not  adequately  factored 

36% 

3.57 

2.23 

p35 

Component  Comptroller  Withhold 

35% 

3.58 

2.34 

p36 

Defense  Contract  Management  Agency  (DCMA) 
administrative  actions 

35% 

3.42 

2.36 

F37 

Redirection  of  contractor  efforts 

35% 

3.47 

2.23 

p38 

OSD  Comptroller  Withhold 

34% 

3.43 

2.37 

R39 

Shortage  of  technical/engineering/test  personnel 

34% 

3.51 

2.17 

p40 

Shortage  of  auditors 

33% 

3.17 

2.43 

F4' 

Slower  burn  rate  than  expected  due  to  unfavorable  Schedule 
Performance  Index 

33% 

3.25 

2.14 

F42 

Awaiting  SAE/CAE-level  review  and  decision 

32% 

3.33 

2.30 

[=43 

SAE/CAE/Component-directed  reprogramming 

32% 

3.27 

2.30 

p44 

Rescission 

32% 

3.16 

2.46 

F45 

Changes  in  systems  specs 

31% 

3.30 

2.03 

p46 

Tenure  of  program  manager  (PM)  and  others  in  key  positions 

31% 

3.11 

2.18 

F47 

Holding  award/incentive  fees  in  commitment  for 
future  obligation 

29% 

3.23 

2.35 

F48 

Inadequate  training 

29% 

3.29 

2.13 

p49 

Shortage  of  managers 

28% 

3.10 

2.17 

F50 

Insufficiently  planned  Overseas  Contingency  Operations 
(OCO)  funding 

27% 

3.07 

2.27 

F5' 

Shortage  of  staff 

26% 

2.99 

2.12 

F52 

Contractor  rework 

26% 

3.00 

2.14 

p53 

Deferred  payments  for  scheduling  earning  fees,  progress 
payments/performance-based  payments 

25% 

3.08 

2.20 

F54 

Effect  of  contract  type  on  outlay  rates 

24% 

2.99 

2.17 

p55 

Materiel/Systems  Command  Comptroller  Withhold 

24% 

2.71 

2.17 

p56 

Awaiting  PEO-level  review  and  decision 

24% 

2.80 

2.01 

F57 

Termination  liability 

22% 

2.72 

2.17 

p58 

Insufficient  workplace  tools/apps 

22% 

2.82 

2.01 

F59 

PEO-directed  programming 

21% 

2.83 

2.10 

p60 

Slower  burn  rate  than  expected  due  to  favorable 

Cost  Performance  Index 

21% 

2.77 

1.95 

F61 

PEO  Withhold 

20% 

2.39 

1.99 

F62 

Preparing  PEO-level  review  and  decision 

20% 

2.66 

1.53 

p63 

Production  line  issues 

19% 

2.82 

2.08 

p64 

Labor  disputes 

10% 

1.89 

1.64 

-la  =  25% 
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FIGURE  3.  IMPACT  RATINGS  ABOVE  X  IN  AGGREGATE 
DESCENDING  ORDER  WITH  RESPONDENT  GROUP  LOW 
AND  HIGH  RATINGS 
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to  reduce  the  adverse  effects  of  all  three  factors,  the  respondents  rec¬ 
ommended  the  “inclusion  of  more  risk  mitigation  into  contract  award 
planning,  more  realistic  timelines,  more  realistic  plans,  greater  funding 
stability,  reduction  in  bureaucratic  obstacles,  more  synchronized  inter¬ 
nal  processes,  and  better  aligned  accounting  systems.” 

Factors  Ranked  One  Standard  Deviation  Above 
the  Mean  (+1?) 

This  next  line  of  demarcation  (Figure  3,  factors  F'’-F9)  included  many 
contracting- related  factors  (i.e..  Shortage  of  contracting  officers  (F4), 
Contractor  proposal  prep  delays  (F6),  Request  for  Proposal  (RFP)  prep 
delays  (F8)  and  Source  selection  delays  (F9).  Nearly  all  the  factors  showed 
the  emergence  of  a  more  alarming  a  between  the  individual  respondent 
groups— as  high  as  18  percent  in  one  case  (i.e.,  Contractor  proposal  prep 
delays  [F6]).  For  this  particular  factor,  procurement  contracting  officers 
(PCO)  reported  the  highest  impact,  while  PMs  ranked  it  as  the  lowest. 
Senior  staff  cited  that  Shortage  of  contracting  officers  (F4)  created  the 
highest  impact,  while  PCOs  reported  it  had  the  lowest  impact.  With  a 
7  percent  a,  it  was  the  lowest  among  all  six  factors  in  this  grouping. 
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Given  that  six  of  the  top  nine  factors  were  contract-specific  factors 
that  ranked  above  +1  a  (Figure  3),  it  came  as  little  surprise  to  see  so  many 
reinforcing  comments  surface. 

•  “Lack  of  experienced  and  qualified  contract  specialists . . 

•  “Alarmingly  low  personnel  qualified . . .  many  unsure/lack 
guidance  and  experience ...” 

•  “Significantly  stressed  with  overtime  to  complete  all  con¬ 
tracting  actions  prior  to  close  of  fiscal  year.” 

•  “Inadequate  training . . .  inordinate  number  of  interns  with 
very  low  experience  in  all  career  fields.” 

•  “Lack  of  sufficient  legal  personnel  trained  in  Acquisition.” 

•  “Loss  in  brain  trust  and  skill  to  develop  complete,  clear 
SOWs  [Statements  of  Work]  using  proactive  contract 
language.” 

•  “SOW  writing  and  the  teaching  of  SOW  writing  classes  is 
greatly  left  to  contractors  or  support  contractors,  resulting 
in  unclear  language.” 

The  highest  frequency  of  occurrence  was  also  associated  with  con¬ 
tracting-related  factors  (Figure  3).  By  far,  the  aggregate  respondents 
rated  Shortage  of  contracting  officers  (F4)  as  the  single  highest  factor 
among  all  22  factors  measured  for  frequency.  Because  the  contracting 
activity  timeline  generally  has  lengthy  durations,  any  disruption  appears 
to  have  an  unmistakable  impact  on  contract  award.  Shortage  of  contract¬ 
ing  officers  (F4)  was  seen  as  having  the  most  significant  impact.  Several 
respondents  said  “multiple  contracting  actions  were  having  compound¬ 
ing  consequences.” 

The  two  remaining  factors  above  +1  a  Congressional  mark  (F5)  and 
OSD -directed  RMD  adjustment  (F7),  had  very  low  frequency  of  occur¬ 
rences,  but  still  reported  a  very  high  impact  similar  to  CRA.  When 
combining  these  with  F4,  all  three  appear  to  be  a  strong  antecedent  force 
(or  moderating  factor)  to  the  already  time-consuming  chain  of  contract¬ 
ing  actions. 
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Factors  Ranked  Above  x 

This  final  grouping  (Figure  3,  factors  F10-F31)  accounted  for  the 
remaining  22  impact  factors.  Perception  polarities  persisted,  especially 
between  two  respondent  groups— senior  staff  outside  the  program  office 
and  PMs  inside  program  offices.  For  PMs  in  every  case  except  one  (i.e.. 
Component-directed  POM  adjustment  [F17]),  the  impact  factors  ranked 
well  below  x.  In  sharp  contrast,  senior  staff,  in  every  case  except  one 
(i.e..  Component-directed  POM  adjustment  [F17]),  stated  the  majority 
of  the  top  31  factors  had  the  largest  impact— or  close  to  it— among  all 
respondent  groups. 

Even  though  the  remaining  impact  factors  above  x  are  still  sig¬ 
nificant,  the  researchers  shifted  the  focus  to  the  presence  of  any  strong 
correlations  since  factor  couplings  could  be  having  a  moderating  effect 
and  require  a  closer  look. 

Factors  That  Correlate 

Table  3  summarizes  the  strongest  and  weakest  factor  correlations 
for  all  respondents  queried.  Several  strong  correlations  surfaced  for 
factors  above  3c.  Changes  in  user  requirements  (F11)  and  Changes  in  user 
priorities  (F19)  were  very  strongly  correlated.  In  three  specific  instances, 
two  factors  above  3c  were  very  strongly  correlated  with  three  factors 
that  fell  below  3c:  Lack  of  experience  levels  in  key  acquisition  functional 
areas  (F27)  and  Inadequate  training  (F48);  Lack  of  experience  levels  in 
key  acquisition  functional  areas  (F27)  and  Tenure  of  PM  and  others  in 
key  positions  (F46);  and  DCMA  administrative  actions  (F36)  and  DCAA 
administrative  actions  (F22).  Three  contract-related  factors  (F4,  F8,  and 
F9)  showed  weaker  correlations  than  expected.  Whether  a  factor  had 
a  weak  correlation  doesn’t  mean  it  had  any  less  importance,  but  any 
course  of  action  intended  to  mitigate  the  presence  of  any  impact  factor 
strongly  correlated  with  another  should  be  weighed  more  heavily  in  any 
recommended  action.  For  example,  the  turnover  of  PMs  could  be  part  of 
the  experience  quotient. 

Factor  Plotting 

The  researchers  generated  a  scatter  plot  diagram  (Figure  4)  that 
punctuated  how  the  31  factors  fluctuated  between  impact  and  frequency 
of  occurrence.  In  some  cases,  the  impact  of  certain  factors  occurred  with 
low  frequencies  of  occurrence.  In  other  cases,  the  frequency  of  occur¬ 
rence  compounded  the  impacts. 
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TABLE  3.  FACTOR  CORRELATION  COUPLING 

Strongest  Correlation  Weakest 

r  r2  Coefficients  Correlation 


Experience,  Training,  and  Tenure: 


.84 

71% 

F27  Key  acquisition  experience 
levels  and  F48  Inadequate  training 

.78 

61% 

F27  Key  acquisition  experience 
levels  and  F46  Tenure  of  PM  and 
other  key  positions 

Administrative  Actions: 

.81 

76% 

F36  DCMA  and  F22  DCAA 

.82 

67% 

F"  User  requirements  and 

F19  User  priorities 

.70 

49% 

F19  User  priorities  and 

F13  Stakeholder  requirements 

Contract-related  Activities: 

.71 

50% 

F6  Contractor  proposal  delay  and 

F2  Contract  negotiations  delays 

.70 

49% 

F3  Contract  award  delays  and 

F2  Contract  negotiations  delays 

The  higher  the  %,  the  stronger  the  direction  and 
strength  of  the  linear  relationship  between  the 
variables 


Factors  #  1  -  3  >  +2o; 
Factors  #  4  -  9  >  +1o 
Factors  #  10  -  31  >  x 


F1  Late  release  of  full  obligation/budget 
authority  due  to  CRA 

F4  Shortage  of  contracting  officers 

F5  Congressional  mark/rescission 

F7  OSD-directed  RMD  adjustment 

F8  RFP  prep  delays 

F'°  Unrealistic/overly  optimistic  spend 
plans 

F12  Changes  to  program  acquisition 
strategy 

F15  Lack  of  decision  authority 

F16  Implementation  of  new  OSD/Service 
policy 

F17  Component-directed  POM  adjustment 

F18  Awaiting  reprogramming  action 

F20Realistic  spend  plans,  but  risks 
materialized 

F21  Program  delays  from  prerequisite 
events 

F23  Unplanned  Congressional  adds  to  PB 
request 

F25 Expenditure  contingent  on  hardware 
delivery 

F26Loss  of  funding  through  reprogramming 

F29Shortage  of  cost  estimators 

F30Shortage  of  business/finance  personnel 

F31  Programmatic  conflicts  between 
government  and  prime  contractor 


The  research  data  results  were  rebased  to  a  Likert-like  scale  for 
plotting  the  frequency  and  adverse  impact  response  averages.  The 
researchers  included  Factors  F29-F31  in  Figure  4  because  they  only  fall 
slightly  below  x. 

For  the  relationships  that  were  co-linear  (e.g.,  the  most  strongly  cor¬ 
related  depicted  in  Table  3),  the  researchers  explored  whether  they  also 
behaved  as  strong  predictors  across  the  sample  population.  After  inves¬ 
tigating  t-ratios  (used  with  ACAT  Level  factors)  and  beta-weights  (used 
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for  the  sample  population),  the  researchers  determined  the  relation¬ 
ships  were  not  significantly  co-linear  enough  to  substantiate  causation. 
Consequently,  there  was  no  merit  in  running  any  further  regression  that 
analyzed  the  factors  as  predictors.  However,  the  researchers  conducted 
another  set  of  tests  by  modulating  certain  respondent  demographics  and 
holding  constant. 


FIGURE  4.  SCATTER  PLOT  OF  IMPACT  FACTORS  WITH  FREQUENCY 


Scatter  Plot  Results  >  x  (for  Impact) 


30 

4 

29 

31 
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1  Factor  listed  by  rating  of 
adverse  Impact,  but  not 
plotted  by  frequency. 
Frequency  data  not 
collected. 


1  Late  release  of  full  obligation/budget  authority  due  to 
CRA 

2  Contract  negotiations  delays 

3  Contract  award  delays 

4  Shortage  of  contracting  officers 

5  Congressional  mark 

6  Contractor  proposal  prep  delays 

7  OSD-directed  RMD  adjustment 

8  RFP  prep  delays 

9  Source  selection  delays 

70  Unrealistic/overly  optimistic  spend 
plans’ 

11  Changes  in  user  requirements 

12  Changes  to  program  acquisition 
strategy' 

13  Changes  in  other  stakeholder  requirements 

14  Preparing  DAE-ievei  review  and 
decision’ 

15  Lack  of  decision  authority  at 
expected  levels’ 

16  Implementation  of  new  OSD/Service  policy 

17  Component-directed  POM  adjustment 

18  Awaiting  reprogramming  action 

19  Changes  in  user  priorities 

20  Realistic  spend  plans,  but  risks 
materialized’ 

21  Program  delays  from  additional  development, 
testing,  or  other  prerequisite  events 

22  DCAA  administrative  actions’ 

23  Unplanned  Congressional  adds  to  PB  request 

24  Use  of  undefinitized  contract  action  delays 

25  Expenditure  contingent  on 
hardware  delivery’ 

26  Loss  of  funding  through  reprogramming  action  to 
higher  priority  requirements  to  PEO  portfolio 

27  Lack  of  experience  levels  in  key 
acquisition  functional  areas’ 

28  Awaiting  DAE-ievei  review  and 
decision’ 

29  Shortage  of  cost  estimators 

30  Shortage  of  business/finance  personnel 

31  Programmatic  conflicts  between  government  and 
prime  contractor 
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Factor  Plotting— Modulating  ACAT  Levels 

Figure  5  shows  how  the  factor  rankings  changed  after  isolating 
ACAT  levels. 


FIGURE  5.  FACTOR  RATINGS  >  x  ACAT  LEVEL 
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MODULATION  OF  FACTORS 

Factor  Decrease: 

F!  Awaiting  reprogramming  action 

F9  Changes  in  user  priorities 

FB  Unplanned  Congressional  adds  to  PB 
request 

F*  Loss  of  funding  through 
reprogramming  action  to  higher 
priority  requirements  to  PEO  portfolio 

Factor  Increase: 

P  Programmatic  conflicts  between 
government  and  prime  contractor 


MODULATION  OF  FACTORS 

Factor  Decrease: 

F2  Contract  negotiation  delays 
F!  Contract  award  delays 
F*  Shortage  of  contracting  officers 
Ff  Contractor  proposal  prep  delays 
P  OSD-directed  RMD  adjustment 
F*  RFP  prep  delays 
F9  Source  selection  delays 
F"  Changes  in  user  requirements 
FB  Changes  in  other  stakeholder 
requirements 

F16  Implementation  of  new  OSD/Service 
policy 

F"  Awaiting  reprogramming  action 
F19  Changes  in  user  priorities 
F21  Program  delays  from  prerequisite 
events 

F23  Unplanned  Congressional  adds  to  PB 
request 

F24  Use  of  undefinitized  contract  action 
delays 


MODULATION  OF  FACTORS 

Factor  Decrease: 

Fl  Implementation  of  new  OSD/Service 
policy 

F‘  Awaiting  reprogramming  action 

F9  Changes  in  user  priorities 

F21  Program  delays  from  prerequisite 
events 

F2i  Unplanned  Congressional  adds  to  PB 
request 

F2<  Use  of  undefinitized  contract  action 
delays 

Factor  Increase: 

F29  Shortage  of  Cost  Estimators 

F!"  Shortage  of  business/finance 
personnel 

F  Redirection  of  contractor  efforts 

F!S  Shortage  of  technical/engineering/ 
testing  personnel 

P  Shortage  of  staff 


ACAT  I.  Funding  and  requirements  factors  (F18,  F19,  F23,  and  F28) 
previously  ranked  above  x  dropped  below  x  while  Contractor  proposal 
prep  delays  (F6)  rose  markedly  to  become  the  highest  impact  factor. 
Component-directed  POM  adjustment  (F17)  made  a  noticeable  shift  to 
the  top  nine  factors  (or  one  standard  deviation  above  the  mean). 
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ACAT  II.  Fifteen  of  the  factors  previously  ranked  above  x  dropped 
below  x  (leaving  only  F1,  F2,  F3,  and  F17).  Four  of  the  factors  that  fell  below 
x  included  contracting-related  factors  (F4,  F6,  F8,  and  F9). 

ACAT  III.  Six  of  the  factors  (F16,  F18,  F19,  F21,  F23,  and  F24)  previously 
ranked  above  x  dropped  below  x.  Shortages  of  personnel  (F29,  F30,  F39,  and 
F51)  and  Redirection  of  contractor  efforts  (F37)  became  more  dominat¬ 
ing  issues  for  the  respondents.  Changes  in  user  priorities  (F19),  Changes 
in  other  stakeholder  requirements  (F13),  and  Loss  of  funding  through 
reprogramming  action  to  higher  priority  requirements  to  PEO  portfolio 
(F26)  all  moved  significantly  above  x. 

What  does  this  mean?  The  more  detailed  differentiation  seen  in  the 
scatter  plots  gives  additional  insight  into  the  factors  that  would  benefit 
from  a  more  focused  investigation  of  each  ACAT.  In  some  cases,  reduc¬ 
ing  frequency  of  occurrence  or  perhaps  instituting  more  early  warning 
metrics  could  have  a  marked  effect  in  reducing  any  adverse  impacts. 

Factor  Plotting— Modulating  Service  Components  and  DoD 

Figure  6  shows  how  the  factor  rankings  changed  after  isolating 
Service  Components. 

U.S.  Army.  No  factors  fell  below x.  The  only  component  where 
factors  moved  above  x  was  Shortage  of  auditors  (F40)  and  Insufficiently 
planned  Overseas  Contingency  Operations  (OCO)  funding  (F50).  Based  on 
historical  information,  OCO  funding  will  most  likely  continue  to  present 
challenges  since  contingency  funding  needs  are  less  predictable  during 
a  wartime  footing. 

U.S.  Air  Force.  Shortage  of  contracting  officers  (F4)  and  Use  of 
undefinitized  contract  action  delays  (F24)  both  dropped  below  x.  Even 
though  Shortage  of  contracting  officers  moved,  there  were  no  companion 
drops  in  contracting-related  factors. 

U.S.  Navy.  Six  factors  dropped  below  x.  Implementation  of  new 
OSD/Service  Policy  (F16),  Awaiting  reprogramming  action  (F18),  Changes 
in  user  priorities  (F19),  Unplanned  Congressional  adds  to  PB  request  (F23), 
Use  of  undefinitized  contract  action  delays  (F24),  and  Loss  of  funding 
through  reprogramming  action  to  higher  priority  requirements  to  PEO 
portfolio  (F26)  became  less  of  an  impact.  For  Navy  respondents,  there  was 
no  notable  movement  in  the  top  six  contracting-related  factor  collective. 
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FIGURE  6.  FACTOR  RATINGS  >x  BY  COMPONENT 
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DoD.  Three  factors  fell  below  x  (i.e.,  Implementation  of  new  OSD/ 
Service  policy  [F16],  Component-directed  POM  adjustment  [F17],  and 
Use  of  undefinitized  contract  action  delays  [F24]),  while  three  factors 
rose  above  x:  OSD  Comptroller  Withhold  (F38),  Shortage  of  business/ 
finance  personnel  (F30),  and  Shortage  of  technical/engineering/test 
personnel  (F39). 

What  does  this  mean?  The  Army  was  the  only  one  of  the  four  group¬ 
ings  that  was  significantly  affected  by  Use  of  undefinitized  contract 
action  delays  (F24);  and  DoD  was  the  only  one  of  the  four  groupings  that 
was  significantly  affected  by  OSD  Comptroller  Withhold  (F38),  Shortage 
of  business/ finance  personnel  (F30),  and  Shortage  of  technical/engineer¬ 
ing/test  personnel  (F39). 

Factor  Plotting— Modulating  Respondent  Groups 

Figure  7  shows  how  the  factor  rankings  changed  after  isolating  the 
respondent  groups. 

Program  Office.  Six  factors  dropped  below x:  Awaiting  repro¬ 
gramming  action  (F18),  Changes  in  user  priorities  (F19),  Program  delays 
from  additional  development,  testing,  or  other  prerequisite  events  (F21), 
Unplanned  Congressional  adds  to  PB  request  (F23),  Use  of  undefinitized 
contract  action  delays  (F24),  and  Loss  of  funding  through  reprogramming 
action  to  higher  priority  requirements  to  PEO  portfolio  (F26).  No  factors 
fell  below  x. 

PEO.  Use  of  undefinitized  contract  action  delays  (F24)  fell  below 
x,  while  four  factors  rose  above  3c:  Shortage  of  cost  estimators  (F29), 
Shortage  of  business/finance  personnel  (F30),  Component  Comptroller 
Withhold  (F35),  and  Insufficiently  planned  OCO  funding  (F50). 

Senior  OSD  Staff.  Awaiting  reprogramming  action  (F18)  fell  below 
x  while  13  factors  rose  above  3c. 

For  PEO  and  senior  OSD  staff,  personnel  shortages  (F29,  F30,  F20,  and 
F40)  became  more  dominant  while  Awaiting  reprogramming  action  (F18) 
became  less  dominant  for  program  office  and  senior  OSD  staff  person¬ 
nel.  Of  the  three  groupings  in  this  particular  case,  nowhere  were  there 
more  factor  increases  than  for  senior  OSD  staff  personnel.  The  rise  in 
Unobligated  prior  year  funding  not  adequately  factored  (F34),  SAE/CAE/ 
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FIGURE  7.  FACTOR  RATINGS  >  JC  WHEN  NEGOTIATED 
CONTRACT  COSTS  WERE  LOWER  THAN  COSTS  PROJECTED 
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Component-directed  reprogramming  (F43),  and  PEO-directed  program¬ 
ming  (F59)  seemed  intuitive  since  senior  staff  may  see  first-hand  the 
longer  time  it  takes  for  program  managers  to  react  to  changes  in  their 
plans.  However,  it  was  very  interesting  to  note  the  disparities  between 
how  senior  OSD  staff  personnel  responded  to  survey  queries  regarding 
the  major  impediments  to  meeting  OSD’s  Obligation  and  Expenditure 
rate  goals  versus  the  responses  from  program  office  personnel,  especially 
shortage  of  personnel  and  contract-specific  factors  (i.e..  Changes  in  sys¬ 
tems  specs  (F45)  and  Redirection  of  contractor  efforts  (F37).  What  does 
this  mean?  This  wide  perception  disparity  deserves  a  more  intensive 
understanding  since  it  could  be  creating  false  perceptions  that  could 
lead  to  misrepresented  positions  and  even  unsubstantiated  decisions. 
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Factor  Plotting— Modulating  Program  Phase 
and  Cost  Projections 

Figure  8  shows  how  the  factor  rankings  changed  after  modulating  by 
program  phase  when  their  negotiated  contract  costs  were  significantly 
lower  than  projections. 


FIGURE  8.  FACTOR  RATINGS  >x  BY  POSITION 
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Development  Phase  (Technology  Development  [TD]  and 
Engineering,  Manufacturing,  and  Development  [EMD]).  Four 
factors  dropped  below  x,  including  Changes  in  other  stakeholder 
requirements  (F13),  Awaiting  reprogramming  action  (F18),  Unplanned 
Congressional  adds  to  PB  request  (F23),  and  Loss  of  funding  through 
reprogramming  action  to  higher  priority  requirements  to  PEO  portfolio 
(F26).  Four  factors  rose  above  x,  including  Shortage  of  business/finance 
personnel  (F30),  Programmatic  conflicts  between  government  and  prime 
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contractor  (F31),  Shortage  of  technical/engineering/test  personnel  (F39), 
and  Holding  award/incentive  fees  in  commitment  for  future  obligation 
(F47).  In  two  cases.  Programmatic  conflicts  between  government  and 
prime  contractor  (F31)  and  Implementation  of  new  OSD  policy  (F16)  made 
a  noticeable  shift  to  the  top  nine  factors  (or  one  standard  deviation  above 
the  mean). 

Procurement  Phase  (Low  Rate  Initial  Production  [LRIP]  and 
Full  Rate  Production  [FRP]).  Eight  of  the  factors  that  previously 
ranked  above  3c  dropped  below  3c.  The  majority  of  the  movement  was 
seen  in  factors  involving  program  delays,  and  funding  and  require¬ 
ments  changes.  The  factors  involving  program  delays  included  Program 
delays  from  additional  development,  testing,  or  other  prerequisite  events 
(F21),  and  Use  of  undefinitized  contract  action  delays  (F24).  The  factors 
involving  funding  delays  included  Unplanned  Congressional  adds  to  PB 
requests  (F23),  and  Awaiting  reprogramming  action  (F18).  The  factors 
involving  requirements  changes  included  Changes  in  user  requirements 
(F11),  Changes  in  other  stakeholder  requirements  (F13),  Changes  in  user 
priorities  (F19),  and  Loss  of  funding  through  reprogramming  action  to 
higher  priority  requirements  to  PEO  portfolio  (F26).  Both  Unobligated 
prior  year  funding  not  adequately  factored  (F34)  and  Shortage  of  techni¬ 
cal,  engineering,  and  test  personnel  (F39)  rose  above  3c. 

In  both  phases,  Changes  in  other  stakeholder  requirements  (F13), 
Awaiting  reprogramming  action  (F18),  Unplanned  Congressional  adds 
to  PB  requests  (F23),  and  Loss  of  funding  through  reprogramming  action 
to  higher  priority  requirements  to  PEO  portfolio  (F28)  fell  below 3c.  In  the 
context  of  modulating  by  program  phase,  the  researchers  found  that  any 
factor  movement  was  negligible  when  costs  met  or  exceeded  projections. 

What  does  this  mean?  Changes  in  user  requirements  (F11)  could 
potentially  be  more  stable  during  the  production  phase  and  no  longer 
become  a  factor.  However,  the  emergence  of  Programmatic  conflicts 
between  government  and  prime  contractor  (F31)  during  the  development 
phase  could  perhaps  be  the  sign  of  competing  motivations  between  DoD 
and  industry  as  well  as  more  prominent  technical  and  schedule  risks. 
All  three  could  result  in  programmatic  delays. 
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Respondent  Comments  Regarding  the  Factors 

The  respondents  were  also  asked  several  open-ended  questions  about 
whether  they  found  the  use  of  metrics  helpful  in  better  meeting  OSD 
goals  as  well  as  any  process  improvements  they  would  recommend.  They 
stated  the  metrics  making  a  difference  for  them  included  “real-time 
monitoring,  frequent  reviews,  tight  coupling  to  contractor  actions  and 
milestones,  and  realistic  spend  plans.”  When  asked  about  any  necessary 
improvements  to  current  processes,  the  respondents  recommended 
including  a  CRA  duration  variable  that  readjusted  expectations, 
establishing  more  realistic  program  goals,  ensuring  more  funding 
stability,  reducing  bureaucratic  obstacles  and  streamlining  more 


FIGURE  9.  SAMPLING  OF  RESPONDENT  COMMENTS 
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outdated  processes,  forging  greater  cooperation  between  government 
and  industry,  and  synchronizing  disparate  accounting  systems  used  in 
Obligation  and  Expenditure  reporting. 

The  respondents  provided  a  number  of  additional  qualitative  com¬ 
ments  that  reinforced  the  quantitative  data,  especially  for  the  factors 
above  >  x  that  were  causing  obligation  rate  interference. 

Recommendations 

What  next?  Based  on  the  research  findings  presented  in  this  article, 
a  number  of  impact  factors  above  x,  if  sufficiently  addressed,  could  help 
lower  the  barriers  to  the  attainment  of  OSD’s  Obligation  and  Expenditure 
rate  goals.  Hence,  the  researchers  offer  the  following  recommendations: 

•  Institute  an  Obligation  and  Expenditure  baseline  adjust¬ 
ment  for  programs  affected  by  any  funding  delay  or 
limitation  (especially  CRA),  then  measure  a  program’s  prog¬ 
ress  to  that  revised  adjustment. 

•  More  thoroughly  review  the  entire  contracting  action  value 
chain.  Look  closely  at  efficiency  opportunities  along  the 
review  and  decision  cycle  continuum,  especially  from  the 
time  an  RFP  is  developed  to  the  time  a  contract  is  let.  Set 
reasonable  time  thresholds  with  triggers  that  afford  more 
proactive  measures  by  PMs— and  confirm  productivity. 

•  Establish  a  recurring  communication  forum  among  key 
stakeholders,  especially  PMs  and  OSD,  to  dialogue  more 
frequently  and  eliminate  perception  gaps  that  could  be 
creating  counterproductive  actions  and  misconceptions. 

•  Track  requirement  changes  throughout  a  program’s  life  and 
look  more  strategically  at  the  effects  on  program  execution 
and  accompanying  Acquisition  Program  Baselines.  Despite 
ACAT  levels,  an  obvious  ripple  effect  is  associated  with  any 
substantive  change  in  program  content  across  a  program’s 
life  that  should  be  codified  more  comprehensively.  However, 
there  are  also  issues  associated  with  different  ACAT  levels, 
which  must  be  noted. 
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Review  the  program  review  cycle  and  streamline  wherever 
possible.  Checks  and  balances  within  the  DoD’s  acquisition 
community  have  always  been  a  vital  constituent  component 
of  program  execution,  but  every  review  should  have  a  dis¬ 
tinctive  purpose,  exit  criteria,  and  associated  suspense  date 
that  is  just  as  material  and  credible. 

Build  and  maintain  realistic  spend  plans,  measure  against 
them,  account  for  contingencies,  and  make  adjustments 
with  required  frequency  due  to  real  world  realities.  Since 
spend  plans  are  subjected  to  so  many  real  world  program¬ 
matic  eventualities,  updating  them  is  vital.  Collaborate  with 
senior  leadership  early  enough  about  required  adjustments 
to  avoid  more  draconian  measures  later. 

Validate  the  key  personnel  shortage  areas  and  recognize  the 
time  it  takes  to  rebuild  those  experience  levels. 

Nurture  experience  in  key  functional  areas  with  strong 
catalysts  such  as  disciplined  on-the-job  training,  programs, 
mentoring,  and  guidance.  With  the  recent  surge  of  contract¬ 
ing  specialist  interns,  their  progress  as  a  group  should  be 
measured  more  carefully. 

Evaluate  the  real  effects  of  reprogramming  action  or 
realignment  of  future  budget  decisions  before  any  correc¬ 
tive  action  is  taken. 

Conduct  a  wholesale  review  of  the  program  execution  met¬ 
rics  currently  in  place  and  determine  their  usefulness  and 
effectiveness.  What  are  they  actually  measuring?  How  are 
these  data  (metrics)  used  and  are  they  worth  collecting? 
Consolidate  whenever  practical  and  eliminate  the  data 
(metrics)  that  have  outlived  their  usefulness. 

Encourage  innovation  and  avoid  the  “bookkeeping  process” 
as  RAND  Corporation  found  in  a  recent  study  that  could  be 
limiting  improvements  championed  by  PMs  (Blickstein  & 
Nemfakos,  2009). 
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Summary 

On  Feb.  5,  2013,  the  authors  presented  the  study  results  discussed 
in  this  article  to  Assistant  Secretary  of  Defense  for  Acquisition 
Katrina  McFarland  and  other  key  OSD  senior  staff.  With  the  metrics 
she  plans  to  institute  with  Better  Buying  Power  (BBP)  2.0,  DoD  will 
have  another  means  to  address  many  of  the  impact  factors  discussed 
herein  and  a  host  of  other  variables  that  could  be  encumbering  pro¬ 
gram  execution  expectations. 

On  Sept.  10,  2012,  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  Frank  Kendall,  and  Under  Secretary  of 
Defense  (Comptroller)  Robert  F.  Hale,  jointly  signed  a  memorandum 
that  listed  six  tenets  that  could  help  combat  some  of  the  same  factors 
discussed  in  this  article  regarding  the  disposition  of  DoD’s  unobligated 
funds  (DoD,  2012).  Over  time,  realization  of  these  tenets  might  also 
reduce  perception  disparity  gaps  among  the  key  personnel  that  have  a 
hand  in  ensuring  our  warfighters  continue  to  get  the  weapon  systems 
they  need— and  on  time— to  best  support  our  national  military  strategy. 
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Review: 

The  Harvard  Business  School’s  J.  Ronald  Fox,  a  long-time  student 
of  acquisition,  prepared  this  volume  drawing  on  work  by  the 
other  contributors.  All  five  have  been  associated  with  the  Defense 
Acquisition  History  Project.  Although  the  book’s  front  matter  implies 
that  the  project  ended  in  2009,  incomplete,  in  fact  it  is  now  housed 
in  the  Historical  Office  of  the  Office  of  the  Secretary  of  Defense  and 
further  volumes  can  be  expected.  This  is  something  to  look  forward 
to,  since  Fox’s  volume  itself  offers  little  that  is  new;  as  a  review  of 
past  studies,  it  will  be  most  useful  to  newcomers  to  the  subject  of 
acquisition  reform. 

There  are  some  fresher  sections.  In  one  of  these.  Fox  and  his  colleagues 
relate  how  the  Air  Force,  Navy,  and  to  a  lesser  extent  the  Army,  sought, 
with  considerable  success,  to  circumvent  or  otherwise  neutralize 
provisions  of  the  1986  Goldwater-Nichols  Act  (see  pp.  127-146).  Mostly, 
however,  and  despite  considerable  use  of  oral  histories  and  internal 
DoD  documents,  Defense  Acquisition  Reform  adds  only  marginally 
to  our  understanding.  This  is  not  so  much  a  criticism  of  the  book  as 
an  acknowledgement  of  how  many  studies  have  gone  over  the  ground 
reviewed,  reaching  many  of  the  same  conclusions. 

What  is  needed  most  is  analytical  insight.  Six  decades  of  attempts  at 
reform  have  largely  failed.  The  message  is  plain  in  Defense  Acquisition 
Reform,  if  largely  implicit,  soft-peddled  even  in  the  subtitle. 

The  book’s  treatment  of  workforce  quality  illustrates  the 
unsatisfactory  state  of  analysis.  The  subject  is  one  that  Fox  has 
examined  previously  and  mentions  repeatedly  here.  It  is  well  and 
good  to  urge  more  and  better  training  of  the  acquisition  workforce, 
stronger  incentives  for  exemplary  performance,  and  lengthier 
tenures,  especially  for  program  managers,  to  build  capability  through 
experience.  But  a  quick  glance  at  the  private  sector  is  enough  to 
show  that  a  skilled  and  experienced  workforce  is  no  assurance  of 
organizational  performance.  For  decades,  U.S.-based  firms  like 
General  Motors  and  IBM  had  their  pick  of  the  best  graduates  of 
the  best  schools.  With  the  help  of  formal  training  and  internal 
labor  markets  that  rewarded  experiential  learning,  they  held  onto 
many  of  these  employees.  IBM,  after  running  into  competitive 
difficulties  some  years  ago,  managed  to  revivify  itself.  But  smart 
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and  capable  employees  were  not  enough  for  GM  to  find  its  way  out 
of  the  organizational  routines  that  entrapped  the  firm  beginning  in 
the  1950s.  Will  GM  finally  make  it  this  time?  How  about  Hewlett- 
Packard?  Sony?  DoD  would  certainly  benefit  from  abetter  qualified 
acquisition  workforce.  Yet  how  much  difference  would  this  actually 
make  for  major  programs  dominated  by  bureaucratic  power  politics? 
The  audience  for  studies  of  acquisition,  certainly  the  policy¬ 
making  audience,  would  benefit  from  attempts  to  answer  questions 
of  this  sort,  no  matter  how  tentative  the  answers  might  be. 
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